Robust Chauvenet Rejection: Powerful, but Easy to Use Outlier Detection for Heavily Contaminated Data Sets [CL]

http://arxiv.org/abs/2301.07838


In Maples et al. (2018) we introduced Robust Chauvenet Outlier Rejection, or RCR, a novel outlier rejection technique that evolves Chauvenet’s Criterion by sequentially applying different measures of central tendency and empirically determining the rejective sigma value. RCR is especially powerful for cleaning heavily-contaminated samples, and unlike other methods such as sigma clipping, it manages to be both accurate and precise when characterizing the underlying uncontaminated distributions of data sets, by using decreasingly robust but increasingly precise statistics in sequence. For this work, we present RCR from a software standpoint, newly implemented as a Python package while maintaining the speed of the C++ original. RCR has been well-tested, calibrated and simulated, and it can be used for both one-dimensional outlier rejection and $n$-dimensional model-fitting, with or without weighted data. RCR is free to use for academic and non-commercial purposes, and the code, documentation and accompanying web calculator can be found and easily used online at https://github.com/nickk124/RCR

Read this paper on arXiv…

N. Konz and D. Reichart
Fri, 20 Jan 23
69/72

Comments: 10 pages, 6 figures, pre-print version. This paper introduces a Python library for the algorithm introduced in arXiv:1807.05276

Towards data-driven modeling and real-time prediction of solar flares and coronal mass ejections [IMA]

http://arxiv.org/abs/2212.14384


Modeling of transient events in the solar atmosphere requires the confluence of 3 critical elements: (1) model sophistication, (2) data availability, and (3) data assimilation. This white paper describes required advances that will enable statistical flare and CME forecasting (e.g. eruption probability and timing, estimation of strength, and CME details, such as speed and magnetic field orientation) similar to weather prediction on Earth.

Read this paper on arXiv…

M. Rempel, Y. Fan, M. Dikpati, et. al.
Mon, 2 Jan 23
14/44

Comments: Heliophysics 2050 White Paper

Microcanonical Hamiltonian Monte Carlo [CL]

http://arxiv.org/abs/2212.08549


We develop Microcanonical Hamiltonian Monte Carlo (MCHMC), a class of models which follow a fixed energy Hamiltonian dynamics, in contrast to Hamiltonian Monte Carlo (HMC), which follows canonical distribution with different energy levels. MCHMC tunes the Hamiltonian function such that the marginal of the uniform distribution on the constant-energy-surface over the momentum variables gives the desired target distribution. We show that MCHMC requires occasional energy conserving billiard-like momentum bounces for ergodicity, analogous to momentum resampling in HMC. We generalize the concept of bounces to a continuous version with partial direction preserving bounces at every step, which gives an energy conserving underdamped Langevin-like dynamics with non-Gaussian noise (MCLMC). MCHMC and MCLMC exhibit favorable scalings with condition number and dimensionality. We develop an efficient hyperparameter tuning scheme that achieves high performance and consistently outperforms NUTS HMC on several standard benchmark problems, in some cases by more than an order of magnitude.

Read this paper on arXiv…

J. Robnik, G. Luca, E. Silverstein, et. al.
Mon, 19 Dec 22
28/62

Comments: 32 pages, 10 figures

Comparison of Step Samplers for Nested Sampling [CL]

http://arxiv.org/abs/2211.09426


Bayesian inference with nested sampling requires a likelihood-restricted prior sampling method, which draws samples from the prior distribution that exceed a likelihood threshold. For high-dimensional problems, Markov Chain Monte Carlo derivatives have been proposed. We numerically study ten algorithms based on slice sampling, hit-and-run and differential evolution algorithms in ellipsoidal, non-ellipsoidal and non-convex problems from 2 to 100 dimensions. Mixing capabilities are evaluated with the nested sampling shrinkage test. This makes our results valid independent of how heavy-tailed the posteriors are. Given the same number of steps, slice sampling is outperformed by hit-and-run and whitened slice sampling, while whitened hit-and-run does not provide as good results. Proposing along differential vectors of live point pairs also leads to the highest efficiencies, and appears promising for multi-modal problems. The tested proposals are implemented in the UltraNest nested sampling package, enabling efficient low and high-dimensional inference of a large class of practical inference problems relevant to astronomy, cosmology, particle physics and astronomy.

Read this paper on arXiv…

J. Buchner
Fri, 18 Nov 22
57/70

Comments: accepted MaxEnt 2022 proceeding, to be published in Physical Sciences Forum. UltraNest nested sampling package this https URL

Nested sampling statistical errors [IMA]

http://arxiv.org/abs/2211.03258


Nested sampling (NS) is a popular algorithm for Bayesian computation. We investigate statistical errors in NS both analytically and numerically. We show two analytic results. First, we show that the leading terms in Skilling’s expression using information theory match the leading terms in Keeton’s expression from an analysis of moments. This approximate agreement was previously only known numerically and was somewhat mysterious. Second, we show that the uncertainty in single NS runs approximately equals the standard deviation in repeated NS runs. Whilst intuitive, this was previously taken for granted. We close by investigating our results and their assumptions in several numerical examples, including cases in which NS uncertainties increase without bound.

Read this paper on arXiv…

A. Fowlie, Q. Li, H. Lv, et. al.
Tue, 8 Nov 22
16/79

Comments: 12 pages + appendices, 3 figures

Nested sampling for physical scientists [CL]

http://arxiv.org/abs/2205.15570


We review Skilling’s nested sampling (NS) algorithm for Bayesian inference and more broadly multi-dimensional integration. After recapitulating the principles of NS, we survey developments in implementing efficient NS algorithms in practice in high-dimensions, including methods for sampling from the so-called constrained prior. We outline the ways in which NS may be applied and describe the application of NS in three scientific fields in which the algorithm has proved to be useful: cosmology, gravitational-wave astronomy, and materials science. We close by making recommendations for best practice when using NS and by summarizing potential limitations and optimizations of NS.

Read this paper on arXiv…

G. Ashton, N. Bernstein, J. Buchner, et. al.
Wed, 1 Jun 22
5/65

Comments: 20 pages + supplementary information, 5 figures. preprint version; published version at this https URL

Machine learning assisted Bayesian model comparison: learnt harmonic mean estimator [CL]

http://arxiv.org/abs/2111.12720


We resurrect the infamous harmonic mean estimator for computing the marginal likelihood (Bayesian evidence) and solve its problematic large variance. The marginal likelihood is a key component of Bayesian model selection since it is required to evaluate model posterior probabilities; however, its computation is challenging. The original harmonic mean estimator, first proposed in 1994 by Newton and Raftery, involves computing the harmonic mean of the likelihood given samples from the posterior. It was immediately realised that the original estimator can fail catastrophically since its variance can become very large and may not be finite. A number of variants of the harmonic mean estimator have been proposed to address this issue although none have proven fully satisfactory. We present the learnt harmonic mean estimator, a variant of the original estimator that solves its large variance problem. This is achieved by interpreting the harmonic mean estimator as importance sampling and introducing a new target distribution. The new target distribution is learned to approximate the optimal but inaccessible target, while minimising the variance of the resulting estimator. Since the estimator requires samples of the posterior only it is agnostic to the strategy used to generate posterior samples. We validate the estimator on a variety of numerical experiments, including a number of pathological examples where the original harmonic mean estimator fails catastrophically. In all cases our learnt harmonic mean estimator is shown to be highly accurate. The estimator is computationally scalable and can be applied to problems of dimension $\mathcal{O}(10^3)$ and beyond. Code implementing the learnt harmonic mean estimator is made publicly available.

Read this paper on arXiv…

J. McEwen, C. Wallis, M. Price, et. al.
Mon, 29 Nov 21
68/94

Comments: 37 pages, 8 figures, code available at this https URL

Astronomical source finding services for the CIRASA visual analytic platform [IMA]

http://arxiv.org/abs/2110.08211


Innovative developments in data processing, archiving, analysis, and visualization are nowadays unavoidable to deal with the data deluge expected in next-generation facilities for radio astronomy, such as the Square Kilometre Array (SKA) and its precursors. In this context, the integration of source extraction and analysis algorithms into data visualization tools could significantly improve and speed up the cataloguing process of large area surveys, boosting astronomer productivity and shortening publication time. To this aim, we are developing a visual analytic platform (CIRASA) for advanced source finding and classification, integrating state-of-the-art tools, such as the CAESAR source finder, the ViaLactea Visual Analytic (VLVA) and Knowledge Base (VLKB). In this work, we present the project objectives and the platform architecture, focusing on the implemented source finding services.

Read this paper on arXiv…

S. Riggia, C. Bordiu, F. Vitello, et. al.
Mon, 18 Oct 21
52/68

Comments: 16 pages, 6 figures

Arby $-$ Fast data-driven surrogates [CL]

http://arxiv.org/abs/2108.01305


The availability of fast to evaluate and reliable predictive models is highly relevant in multi-query scenarios where evaluating some quantities in real, or near-real-time becomes crucial. As a result, reduced-order modelling techniques have gained traction in many areas in recent years. We introduce Arby, an entirely data-driven Python package for building reduced order or surrogate models. In contrast to standard approaches, which involve solving partial differential equations, Arby is entirely data-driven. The package encompasses several tools for building and interacting with surrogate models in a user-friendly manner. Furthermore, fast model evaluations are possible at a minimum computational cost using the surrogate model. The package implements the Reduced Basis approach and the Empirical Interpolation Method along a classic regression stage for surrogate modelling. We illustrate the simplicity in using Arby to build surrogates through a simple toy model: a damped pendulum. Then, for a real case scenario, we use Arby to describe CMB temperature anisotropies power spectra. On this multi-dimensional setting, we find that out from an initial set of $80,000$ power spectra solutions with $3,000$ multipole indices each, could be well described at a given tolerance error, using just a subset of $84$ solutions.

Read this paper on arXiv…

A. Villanueva, M. Beroiz, J. Cabral, et. al.
Wed, 4 Aug 21
49/66

Comments: 10 pages, 8 figures

Systematic evaluation of variability detection methods for eROSITA [HEAP]

http://arxiv.org/abs/2106.14529


The reliability of detecting source variability in sparsely and irregularly sampled X-ray light curves is investigated. This is motivated by the unprecedented survey capabilities of eROSITA onboard SRG, providing light curves for many thousand sources in its final-depth equatorial deep field survey. Four methods for detecting variability are evaluated: excess variance, amplitude maximum deviations, Bayesian blocks and a new Bayesian formulation of the excess variance. We judge the false detection rate of variability based on simulated Poisson light curves of constant sources, and calibrate significance thresholds. Simulations with flares injected favour the amplitude maximum deviation as most sensitive at low false detections. Simulations with white and red stochastic source variability favour Bayesian methods. The results are applicable also for the million sources expected in eROSITA’s all-sky survey.

Read this paper on arXiv…

J. Buchner, T. Boller, D. Bogensberger, et. al.
Tue, 29 Jun 21
64/101

Comments: Resubmitted version after a positive first referee report. Variability analysis tools available this https URL 15 min Talk: this https URL To appear on A&A, Special Issue: The Early Data Release of eROSITA and Mikhail Pavlinsky ART-XC on the SRG Mission

Fasano-Franceschini Test: an Implementation of a 2-Dimensional Kolmogorov-Smirnov test in R [CL]

http://arxiv.org/abs/2106.10539


The univariate Kolmogorov-Smirnov (KS) test is a non-parametric statistical test designed to assess whether a set of data is consistent with a given probability distribution (or, in the two-sample case, whether the two samples come from the same underlying distribution). The versatility of the KS test has made it a cornerstone of statistical analysis and is commonly used across the scientific disciplines. However, the test proposed by Kolmogorov and Smirnov does not naturally extend to multidimensional distributions. Here, we present the fasano.franceschini.test package, an R implementation of the 2-D KS two-sample test as defined by Fasano and Franceschini (Fasano and Franceschini 1987). The fasano.franceschini.test package provides three improvements over the current 2-D KS test on the Comprehensive R Archive Network (CRAN): (i) the Fasano and Franceschini test has been shown to run in $O(n^2)$ versus the Peacock implementation which runs in $O(n^3)$; (ii) the package implements a procedure for handling ties in the data; and (iii) the package implements a parallelized bootstrapping procedure for improved significance testing. Ultimately, the fasano.franceschini.test package presents a robust statistical test for analyzing random samples defined in 2-dimensions.

Read this paper on arXiv…

E. Ness-Cohn and R. Braun
Tue, 22 Jun 21
71/71

Comments: 8 pages, 4 figures

Nested sampling for frequentist computation: fast estimation of small $p$-values [CL]

http://arxiv.org/abs/2105.13923


We propose a novel method for computing $p$-values based on nested sampling (NS) applied to the sampling space rather than the parameter space of the problem, in contrast to its usage in Bayesian computation. The computational cost of NS scales as $\log^2{1/p}$, which compares favorably to the $1/p$ scaling for Monte Carlo (MC) simulations. For significances greater than about $4\sigma$ in both a toy problem and a simplified resonance search, we show that NS requires orders of magnitude fewer simulations than ordinary MC estimates. This is particularly relevant for high-energy physics, which adopts a $5\sigma$ gold standard for discovery. We conclude with remarks on new connections between Bayesian and frequentist computation and possibilities for tuning NS implementations for still better performance in this setting.

Read this paper on arXiv…

A. Fowlie, S. Hoof and W. Handley
Mon, 31 May 21
25/72

Comments: 6 pages, 3 figures

GPU-Accelerated Hierarchical Bayesian Inference with Application to Modeling Cosmic Populations: CUDAHM [IMA]

http://arxiv.org/abs/2105.08026


We describe a computational framework for hierarchical Bayesian inference with simple (typically single-plate) parametric graphical models that uses graphics processing units (GPUs) to accelerate computations, enabling deployment on very large datasets. Its C++ implementation, CUDAHM (CUDA for Hierarchical Models) exploits conditional independence between instances of a plate, facilitating massively parallel exploration of the replication parameter space using the single instruction, multiple data architecture of GPUs. It provides support for constructing Metropolis-within-Gibbs samplers that iterate between GPU-accelerated robust adaptive Metropolis sampling of plate-level parameters conditional on upper-level parameters, and Metropolis-Hastings sampling of upper-level parameters on the host processor conditional on the GPU results. CUDAHM is motivated by demographic problems in astronomy, where density estimation and linear and nonlinear regression problems must be addressed for populations of thousands to millions of objects whose features are measured with possibly complex uncertainties. We describe a thinned latent point process framework for modeling such demographic data. We demonstrate accurate GPU-accelerated parametric conditional density deconvolution for simulated populations of up to 300,000 objects in ~1 hour using a single NVIDIA Tesla K40c GPU. Supplementary material provides details about the CUDAHM API and the demonstration problem.

Read this paper on arXiv…

J. Szalai-Gindl, T. Loredo, B. Kelly, et. al.
Tue, 18 May 21
34/77

Comments: 28 pages, 7 figures, 2 appendices

Improving exoplanet detection capabilities with the false inclusion probability. Comparison with other detection criteria in the context of radial velocities [EPA]

http://arxiv.org/abs/2105.06995


Context. In exoplanet searches with radial velocity data, the most common statistical significance metrics are the Bayes factor and the false alarm probability (FAP). Both have proved useful, but do not directly address whether an exoplanet detection should be claimed. Furthermore, it is unclear which detection threshold should be taken and how robust the detections are to model misspecification. Aims. The present work aims at defining a detection criterion which conveys as precisely as possible the information needed to claim an exoplanet detection. We compare this new criterion to existing ones in terms of sensitivity and robustness. Methods. We define a significance metric called the false inclusion probability (FIP) based on the posterior probability of presence of a planet. Posterior distributions are computed with the nested sampling package Polychord. We show that for FIP and Bayes factor calculations, defining priors on linear parameters as Gaussian mixture models allows to significantly speed up computations. The performances of the FAP, Bayes factor and FIP are studied with simulations as well as analytical arguments. We compare the methods assuming the model is correct, then evaluate their sensitivity to the prior and likelihood choices. Results. Among other properties, the FIP offers ways to test the reliability of the significance levels, it is particularly efficient to account for aliasing and allows to exclude the presence of planets with a certain confidence. We find that, in our simulations, the FIP outperforms existing detection metrics. We show that planet detections are sensitive to priors on period and semi-amplitude and that letting free the noise parameters offers better performances than fixing a noise model based on a fit to ancillary indicators.

Read this paper on arXiv…

N. Hara, N. Unger, J. Delisle, et. al.
Mon, 17 May 21
14/55

Comments: Accepted for publication in Astronomy & Astrophysics

Model-based clustering of partial records [CL]

http://arxiv.org/abs/2103.16336


Partially recorded data are frequently encountered in many applications. In practice, such datasets are usually clustered by removing incomplete cases or features with missing values, or by imputing missing values, followed by application of a clustering algorithm to the resulting altered data set. Here, we develop clustering methodology through a model-based approach using the marginal density for the observed values, using a finite mixture model of multivariate $t$ distributions. We compare our algorithm to the corresponding full expectation-maximization (EM) approach that considers the missing values in the incomplete data set and makes a missing at random (MAR) assumption, as well as case deletion and imputation. Since only the observed values are utilized, our approach is computationally more efficient than imputation or full EM. Simulation studies demonstrate that our approach has favorable recovery of the true cluster partition compared to case deletion and imputation under various missingness mechanisms, and is more robust to extreme MAR violations than the full EM approach since it does not use the observed values to inform those that are missing. Our methodology is demonstrated on a problem of clustering gamma-ray bursts and is implemented in the this https URL R package.

Read this paper on arXiv…

E. Goren and R. Maitra
Wed, 31 Mar 2021
48/62

Comments: 13 pages, 3 figures, 1 table

Nested sampling with any prior you like [IMA]

http://arxiv.org/abs/2102.12478


Nested sampling is an important tool for conducting Bayesian analysis in Astronomy and other fields, both for sampling complicated posterior distributions for parameter inference, and for computing marginal likelihoods for model comparison. One technical obstacle to using nested sampling in practice is the requirement that prior distributions be provided in the form of bijective transformations from the unit hyper-cube to the target prior density. For many applications – particularly when using the posterior from one experiment as the prior for another – such a transformation is not readily available. In this letter we show that parametric bijectors trained on samples from a desired prior density provide a general-purpose method for constructing transformations from the uniform base density to a target prior, enabling the practical use of nested sampling under arbitrary priors. We demonstrate the use of trained bijectors in conjunction with nested sampling on a number of examples from cosmology.

Read this paper on arXiv…

J. Alsing and W. Handley
Fri, 26 Feb 21
34/60

Comments: 5 pages, 2 figures, prepared for submission as an MNRAS letter

UltraNest — a robust, general purpose Bayesian inference engine [CL]

http://arxiv.org/abs/2101.09604


UltraNest is a general-purpose Bayesian inference package for parameter estimation and model comparison. It allows fitting arbitrary models specified as likelihood functions written in Python, C, C++, Fortran, Julia or R. With a focus on correctness and speed (in that order), UltraNest is especially useful for multi-modal or non-Gaussian parameter spaces, computational expensive models, in robust pipelines. Parallelisation to computing clusters and resuming incomplete runs is available.

Read this paper on arXiv…

J. Buchner
Tue, 26 Jan 21
72/84

Comments: Longer version of the paper submitted to JOSS. UltraNest can be found at this https URL

Nested Sampling Methods [CL]

http://arxiv.org/abs/2101.09675


Nested sampling (NS) computes parameter posterior distributions and makes Bayesian model comparison computationally feasible. Its strengths are the unsupervised navigation of complex, potentially multi-modal posteriors until a well-defined termination point. A systematic literature review of nested sampling algorithms and variants is presented. We focus on complete algorithms, including solutions to likelihood-restricted prior sampling. A new formulation of NS is presented, which casts the parameter space exploration as a search on a tree. Previously published ways of obtaining robust error estimates and dynamic variations of the number of live points are presented as special cases of this formulation.

Read this paper on arXiv…

J. Buchner
Tue, 26 Jan 21
84/84

Comments: Comments are welcome. The open-source UltraNest package and astrostatistics tutorials can be found at this https URL

BAT.jl — A Julia-based tool for Bayesian inference [CL]

http://arxiv.org/abs/2008.03132


We describe the development of a multi-purpose software for Bayesian statistical inference, BAT.jl, written in the Julia language. The major design considerations and implemented algorithms are summarized here, together with a test suite that ensures the proper functioning of the algorithms. We also give an extended example from the realm of physics that demonstrates the functionalities of BAT.jl.

Read this paper on arXiv…

O. Schulz, F. Beaujean, A. Caldwell, et. al.
Mon, 10 Aug 20
-773/53

Comments: N/A

Nested sampling cross-checks using order statistics [CL]

http://arxiv.org/abs/2006.03371


Nested sampling (NS) is an invaluable tool in data analysis in modern astrophysics, cosmology, gravitational wave astronomy and particle physics. We identify a previously unused property of NS related to order statistics: the insertion indexes of new live points into the existing live points should be uniformly distributed. This observation enabled us to create a novel cross-check of single NS runs. The tests can detect when an NS run failed to sample new live points from the constrained prior and plateaus in the likelihood function, which break an assumption of NS and thus leads to unreliable results. We applied our cross-check to NS runs on toy functions with known analytic results in 2 – 50 dimensions, showing that our approach can detect problematic runs on a variety of likelihoods, settings and dimensions. As an example of a realistic application, we cross-checked NS runs performed in the context of cosmological model selection. Since the cross-check is simple, we recommend that it become a mandatory test for every applicable NS run.

Read this paper on arXiv…

A. Fowlie, W. Handley and L. Su
Mon, 8 Jun 20
47/57

Comments: 8 pages, 1 figure

Data Analysis Recipes: Products of multivariate Gaussians in Bayesian inferences [CL]

http://arxiv.org/abs/2005.14199


A product of two Gaussians (or normal distributions) is another Gaussian. That’s a valuable and useful fact! Here we use it to derive a refactoring of a common product of multivariate Gaussians: The product of a Gaussian likelihood times a Gaussian prior, where some or all of those parameters enter the likelihood only in the mean and only linearly. That is, a linear, Gaussian, Bayesian model. This product of a likelihood times a prior pdf can be refactored into a product of a marginalized likelihood (or a Bayesian evidence) times a posterior pdf, where (in this case) both of these are also Gaussian. The means and variance tensors of the refactored Gaussians are straightforward to obtain as closed-form expressions; here we deliver these expressions, with discussion. The closed-form expressions can be used to speed up and improve the precision of inferences that contain linear parameters with Gaussian priors. We connect these methods to inferences that arise frequently in physics and astronomy.
If all you want is the answer, the question is posed and answered at the beginning of Section 3. We show two toy examples, in the form of worked exercises, in Section 4. The solutions, discussion, and exercises in this Note are aimed at someone who is already familiar with the basic ideas of Bayesian inference and probability.

Read this paper on arXiv…

D. Hogg, A. Price-Whelan and B. Leistedt
Mon, 1 Jun 20
3/50

Comments: a chapter of a book we will never write

Ridges in the Dark Energy Survey for cosmic trough identification [CEA]

http://arxiv.org/abs/2005.08583


Cosmic voids and their corresponding redshift-aggregated projections of mass densities, known as troughs, play an important role in our attempt to model the large-scale structure of the Universe. Understanding these structures leads to tests comparing the standard model with alternative cosmologies, constraints on the dark energy equation of state, and provides evidence to differentiate among gravitational theories. In this paper, we extend the subspace-constrained mean shift algorithm, a recently introduced method to estimate density ridges, and apply it to 2D weak-lensing mass density maps from the Dark Energy Survey Y1 data release to identify curvilinear filamentary structures. We compare the obtained ridges with previous approaches to extract trough structure in the same data, and apply curvelets as an alternative wavelet-based method to constrain densities. We then invoke the Wasserstein distance between noisy and noiseless simulations to validate the denoising capabilities of our method. Our results demonstrate the viability of ridge estimation as a precursor for denoising weak lensing quantities to recover the large-scale structure, paving the way for a more versatile and effective search for troughs.

Read this paper on arXiv…

B. Moews, M. Schmitz, A. Lawler, et. al.
Tue, 19 May 20
87/92

Comments: 12 pages, 5 figures, preprint submitted to MNRAS

Efficient modeling of correlated noise II. A flexible noise model with fast and scalable methods [IMA]

http://arxiv.org/abs/2004.10678


Correlated noise affects most astronomical datasets and to neglect accounting for it can lead to spurious signal detections, especially in low signal-to-noise conditions, which is often the context in which new discoveries are pursued. For instance, in the realm of exoplanet detection with radial velocity time series, stellar variability can induce false detections. However, a white noise approximation is often used because accounting for correlated noise when analyzing data implies a more complex analysis. Moreover, the computational cost can be prohibitive as it typically scales as the cube of the dataset size.
For some restricted classes of correlated noise models, there are specific algorithms that can be used to help bring down the computational cost. This improvement in speed is particularly useful in the context of Gaussian process regression, however, it comes at the expense of the generality of the noise model.
Here, we present the S+LEAF noise model, which allows us to account for a large class of correlated noises with a linear scaling of the computational cost with respect to the size of the dataset. The S+LEAF model includes, in particular, mixtures of quasiperiodic kernels and calibration noise. This efficient modeling is made possible by a sparse representation of the covariance matrix of the noise and the use of dedicated algorithms for matrix inversion, solving, determinant computation, etc.
We applied the S+LEAF model to reanalyze the HARPS radial velocity time series of HD 136352. We illustrate the flexibility of the S+LEAF model in handling various sources of noise. We demonstrate the importance of taking correlated noise into account, and especially calibration noise, to correctly assess the significance of detected signals.
We provide an open-source implementation of the S+LEAF model, available at https://gitlab.unige.ch/jean-baptiste.delisle/spleaf.

Read this paper on arXiv…

J. Delisle, N. Hara and D. Ségransan
Thu, 23 Apr 20
45/45

Comments: Accepted in A&A

Inpainting via Generative Adversarial Networks for CMB data analysis [CEA]

http://arxiv.org/abs/2004.04177


In this work, we propose a new method to inpaint the CMB signal in regions masked out following a point source extraction process. We adopt a modified Generative Adversarial Network (GAN) and compare different combinations of internal (hyper-)parameters and training strategies. We study the performance using a suitable $\mathcal{C}_r$ variable in order to estimate the performance regarding the CMB power spectrum recovery. We consider a test set where one point source is masked out in each sky patch with a 1.83 $\times$ 1.83 squared degree extension, which, in our gridding, corresponds to 64 $\times$ 64 pixels. The GAN is optimized for estimating performance on Planck 2018 total intensity simulations. The training makes the GAN effective in reconstructing a masking corresponding to about 1500 pixels with $1\%$ error down to angular scales corresponding to about 5 arcminutes.

Read this paper on arXiv…

A. Sadr and F. Farsian
Fri, 10 Apr 20
55/56

Comments: 19 pages, 21 figures. Prepared for submission to JCAP. All codes will be published after acceptance

Clustering of Local Extrema in Planck CMB maps [CEA]

http://arxiv.org/abs/2003.07364


The clustering of local extrema including peaks and troughs will be exploited to assess Gaussianity, asymmetry and the footprint of cosmic strings network on the CMB random field observed by {\it Planck} satellite. The number density of local extrema reveals some non-resolved shot noise in {\it Planck} maps. The \texttt{SEVEM} data has maximum number density of peaks, $n_{pk}$, and troughs, $n_{tr}$, compared to other observed maps. The cumulative of $n_{pk}$ and $n_{tr}$ above and below a threshold, $\vartheta$, for all {\it Planck} maps except for the 100GHz band are compatible with the Gaussian random field. The unweighted Two-Point Correlation Function (TPCF), $\Psi(\theta;\vartheta)$, of the local extrema illustrates significant non-Gaussianity for angular separation $\theta\le 15’$ for all available thresholds. Our results show that to put the feasible constraint on the amplitude of the mass function based on the value of $\Psi$ around the {\it Doppler peak} ($\theta\approx 70′-75’$), we should consider $\vartheta\gtrsim+1.0$. The scale independent bias factors for peak above a threshold for large separation angle and high threshold level are in agreement with that of expected for a pure Gaussian CMB. Unweighted TPCF of local extrema demonstrates a level of rejecting Gaussian hypothesis in \texttt{SMICA}. Genus topology also confirms the Gaussian hypothesis for different component separation maps. Tessellating CMB map with disk of size $6^{\circ}$ based on $n_{pk}$ and $\Psi_{pk-pk}$ demonstrate statistical symmetry in {\it Planck} maps. Combining all maps and applying the $\Psi_{pk-pk}$ puts the upper bound on the cosmic string’s tension: $G\mu^{(up)} \lesssim 5.00\times 10^{-7}$.

Read this paper on arXiv…

A. Sadr and S. Movahed
Wed, 18 Mar 20
19/46

Comments: 16 pages, 19 figures

Ensemble Slice Sampling [CL]

http://arxiv.org/abs/2002.06212


Slice Sampling has emerged as a powerful Markov Chain Monte Carlo algorithm that adapts to the characteristics of the target distribution with minimal hand-tuning. However, Slice Sampling’s performance is highly sensitive to the user-specified initial length scale hyperparameter. Moreover, Slice Sampling generally struggles with poorly scaled or strongly correlated distributions. This paper introduces Ensemble Slice Sampling, a new class of algorithms that bypasses such difficulties by adaptively tuning the length scale. Furthermore, Ensemble Slice Sampling’s performance is immune to linear correlations by exploiting an ensemble of parallel walkers. These algorithms are trivial to construct, require no hand-tuning, and can easily be implemented in parallel computing environments. Empirical tests show that Ensemble Slice Sampling can improve efficiency by more than an order of magnitude compared to conventional MCMC methods on highly correlated target distributions such as the Autoregressive Process of Order 1 and the Correlated Funnel distribution.

Read this paper on arXiv…

M. Karamanis and F. Beutler
Tue, 18 Feb 20
53/72

Comments: N/A

Geometric nested sampling: sampling from distributions defined on non-trivial geometries [CL]

http://arxiv.org/abs/2002.04123


Metropolis Hastings nested sampling evolves a Markov chain, accepting new points along the chain according to a version of the Metropolis Hastings acceptance ratio, which has been modified to satisfy the nested sampling likelihood constraint. The geometric nested sampling algorithm I present here is based on the Metropolis Hastings method, but treats parameters as though they represent points on certain geometric objects, namely circles, tori and spheres. For parameters which represent points on a circle or torus, the trial distribution is “wrapped” around the domain of the posterior distribution such that samples cannot be rejected automatically when evaluating the Metropolis ratio due to being outside the sampling domain. Furthermore, this enhances the mobility of the sampler. For parameters which represent coordinates on the surface of a sphere, the algorithm transforms the parameters into a Cartesian coordinate system before sampling which again makes sure no samples are automatically rejected, and provides a physically intuitive way of the sampling the parameter space.

Read this paper on arXiv…

K. Javid
Wed, 12 Feb 20
53/58

Comments: Peer reviewed and published in JOSS. arXiv admin note: substantial text overlap with arXiv:1905.09110

Mean shift cluster recognition method implementation in the nested sampling algorithm [CL]

http://arxiv.org/abs/2002.01431


Nested sampling is an efficient algorithm for the calculation of the Bayesian evidence and posterior parameter probability distributions. It is based on the step-by-step exploration of the parameter space by Monte Carlo sampling with a series of values sets called live points that evolve towards the region of interest, i.e. where the likelihood function is maximal. In presence of several local likelihood maxima, the algorithm converges with difficulty. Some systematic errors can also be introduced by unexplored parameter volume regions. In order to avoid this, different methods are proposed in the literature for an efficient search of new live points, even in presence of local maxima. Here we present a new solution based on the mean shift cluster recognition method implemented in a random walk search algorithm. The clustering recognition is integrated within the Bayesian analysis program NestedFit. It is tested with the analysis of some difficult cases. Compared to the analysis results without cluster recognition, the computation time is considerably reduced. At the same time, the entire parameter space is efficiently explored, which translates into a smaller uncertainty of the extracted value of the Bayesian evidence.

Read this paper on arXiv…

M. Trassinelli and P. Ciccodicola
Wed, 5 Feb 20
17/67

Comments: N/A

Dynamic Gauss Newton Metropolis Algorithm [CL]

http://arxiv.org/abs/2001.03530


GNM: The MCMC Jagger. A rocking awesome sampler. This python package is an affine invariant Markov chain Monte Carlo (MCMC) sampler based on the dynamic Gauss-Newton-Metropolis (GNM) algorithm. The GNM algorithm is specialized in sampling highly non-linear posterior probability distribution functions of the form $e^{-||f(x)||^2/2}$, and the package is an implementation of this algorithm. On top of the back-off strategy in the original GNM algorithm, there is the dynamic hyper-parameter optimization feature added to the algorithm and included in the package to help increase performance of the back-off and therefore the sampling. Also, there are the Jacobian tester, error bars creator and many more features for the ease of use included in the code. The problem is introduced and a guide to installation is given in the introduction. Then how to use the python package is explained. The algorithm is given and finally there are some examples using exponential time series to show the performance of the algorithm and the back-off strategy.

Read this paper on arXiv…

M. Ugurbil
Mon, 13 Jan 20
1/61

Comments: 21 pages, 5 figures

Normalizing Constant Estimation with Gaussianized Bridge Sampling [CL]

http://arxiv.org/abs/1912.06073


Normalizing constant (also called partition function, Bayesian evidence, or marginal likelihood) is one of the central goals of Bayesian inference, yet most of the existing methods are both expensive and inaccurate. Here we develop a new approach, starting from posterior samples obtained with a standard Markov Chain Monte Carlo (MCMC). We apply a novel Normalizing Flow (NF) approach to obtain an analytic density estimator from these samples, followed by Optimal Bridge Sampling (OBS) to obtain the normalizing constant. We compare our method which we call Gaussianized Bridge Sampling (GBS) to existing methods such as Nested Sampling (NS) and Annealed Importance Sampling (AIS) on several examples, showing our method is both significantly faster and substantially more accurate than these methods, and comes with a reliable error estimation.

Read this paper on arXiv…

H. Jia and U. Seljak
Fri, 13 Dec 19
70/75

Comments: Accepted by AABI 2019 Proceedings

emcee v3: A Python ensemble sampling toolkit for affine-invariant MCMC [IMA]

http://arxiv.org/abs/1911.07688


emcee is a Python library implementing a class of affine-invariant ensemble samplers for Markov chain Monte Carlo (MCMC). This package has been widely applied to probabilistic modeling problems in astrophysics where it was originally published, with some applications in other fields. When it was first released in 2012, the interface implemented in emcee was fundamentally different from the MCMC libraries that were popular at the time, such as PyMC, because it was specifically designed to work with “black box” models instead of structured graphical models. This has been a popular interface for applications in astrophysics because it is often non-trivial to implement realistic physics within the modeling frameworks required by other libraries. Since emcee’s release, other libraries have been developed with similar interfaces, such as dynesty (Speagle 2019). The version 3.0 release of emcee is the first major release of the library in about 6 years and it includes a full re-write of the computational backend, several commonly requested features, and a set of new “move” implementations.

Read this paper on arXiv…

D. Foreman-Mackey, W. Farr, M. Sinha, et. al.
Tue, 19 Nov 19
53/65

Comments: Published in the Journal for Open Source Software

A Flexible Framework for Anomaly Detection via Dimensionality Reduction [CL]

http://arxiv.org/abs/1909.04060


Anomaly detection is challenging, especially for large datasets in high dimensions. Here we explore a general anomaly detection framework based on dimensionality reduction and unsupervised clustering. We release DRAMA, a general python package that implements the general framework with a wide range of built-in options. We test DRAMA on a wide variety of simulated and real datasets, in up to 3000 dimensions, and find it robust and highly competitive with commonly-used anomaly detection algorithms, especially in high dimensions. The flexibility of the DRAMA framework allows for significant optimization once some examples of anomalies are available, making it ideal for online anomaly detection, active learning and highly unbalanced datasets.

Read this paper on arXiv…

A. Sadr, B. Bassett and M. Kunz
Thu, 19 Sep 19
45/71

Comments: 6 pages

CAESAR source finder: recent developments and testing [IMA]

http://arxiv.org/abs/1909.06116


A new era in radioastronomy will begin with the upcoming large-scale surveys planned at the Australian Square Kilometre Array Pathfinder (ASKAP). ASKAP started its Early Science program in October 2017 and several target fields were observed during the array commissioning phase. The SCORPIO field was the first observed in the Galactic Plane in Band 1 (792-1032 MHz) using 15 commissioned antennas. The achieved sensitivity and large field of view already allow to discover new sources and survey thousands of existing ones with improved precision with respect to previous surveys. Data analysis is currently ongoing to deliver the first source catalogue. Given the increased scale of the data, source extraction and characterization, even in this Early Science phase, have to be carried out in a mostly automated way. This process presents significant challenges due to the presence of extended objects and diffuse emission close to the Galactic Plane. In this context we have extended and optimized a novel source finding tool, named CAESAR , to allow extraction of both compact and extended sources from radio maps. A number of developments have been done driven by the analysis of the SCORPIO map and in view of the future ASKAP Galactic Plane survey. The main goals are the improvement of algorithm performances and scalability as well as of software maintainability and usability within the radio community. In this paper we present the current status of CAESAR and report a first systematic characterization of its performance for both compact and extended sources using simulated maps. Future prospects are discussed in light of the obtained results.

Read this paper on arXiv…

S. Riggi, F. Vitello, U. Becciani, et. al.
Mon, 16 Sep 19
8/74

Comments: 15 pages, 10 figures

Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference [IMA]

http://arxiv.org/abs/1908.11523


It is well known in astronomy that propagating non-Gaussian prediction uncertainty in photometric redshift estimates is key to reducing bias in downstream cosmological analyses. Similarly, likelihood-free inference approaches, which are beginning to emerge as a tool for cosmological analysis, require the full uncertainty landscape of the parameters of interest given observed data. However, most machine learning (ML) based methods with open-source software target point prediction or classification, and hence fall short in quantifying uncertainty in complex regression and parameter inference settings such as the applications mentioned above. As an alternative to methods that focus on predicting the response (or parameters) $\mathbf{y}$ from features $\mathbf{x}$, we provide nonparametric conditional density estimation (CDE) tools for approximating and validating the entire probability density $\mathrm{p}(\mathbf{y} \mid \mathbf{x})$ given training data for $\mathbf{x}$ and $\mathbf{y}$. As there is no one-size-fits-all CDE method, the goal of this work is to provide a comprehensive range of statistical tools and open-source software for nonparametric CDE and method assessment which can accommodate different types of settings and which in addition can easily be fit to the problem at hand. Specifically, we introduce CDE software packages in $\texttt{Python}$ and $\texttt{R}$ based on four ML prediction methods adapted and optimized for CDE: $\texttt{NNKCDE}$, $\texttt{RFCDE}$, $\texttt{FlexCode}$, and $\texttt{DeepCDE}$. Furthermore, we present the $\texttt{cdetools}$ package, which includes functions for computing a CDE loss function for model selection and tuning of parameters, together with diagnostics functions. We provide sample code in $\texttt{Python}$ and $\texttt{R}$ as well as examples of applications to photometric redshift estimation and likelihood-free cosmology via CDE.

Read this paper on arXiv…

N. Dalmasso, T. Pospisil, A. Lee, et. al.
Mon, 2 Sep 19
60/66

Comments: 23 pages, 4 figures, 3 tables

Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference [IMA]

http://arxiv.org/abs/1908.11523


It is well known in astronomy that propagating non-Gaussian prediction uncertainty in photometric redshift estimates is key to reducing bias in downstream cosmological analyses. Similarly, likelihood-free inference approaches, which are beginning to emerge as a tool for cosmological analysis, require the full uncertainty landscape of the parameters of interest given observed data. However, most machine learning (ML) based methods with open-source software target point prediction or classification, and hence fall short in quantifying uncertainty in complex regression and parameter inference settings such as the applications mentioned above. As an alternative to methods that focus on predicting the response (or parameters) $\mathbf{y}$ from features $\mathbf{x}$, we provide nonparametric conditional density estimation (CDE) tools for approximating and validating the entire probability density $\mathrm{p}(\mathbf{y} \mid \mathbf{x})$ given training data for $\mathbf{x}$ and $\mathbf{y}$. As there is no one-size-fits-all CDE method, the goal of this work is to provide a comprehensive range of statistical tools and open-source software for nonparametric CDE and method assessment which can accommodate different types of settings and which in addition can easily be fit to the problem at hand. Specifically, we introduce CDE software packages in $\texttt{Python}$ and $\texttt{R}$ based on four ML prediction methods adapted and optimized for CDE: $\texttt{NNKCDE}$, $\texttt{RFCDE}$, $\texttt{FlexCode}$, and $\texttt{DeepCDE}$. Furthermore, we present the $\texttt{cdetools}$ package, which includes functions for computing a CDE loss function for model selection and tuning of parameters, together with diagnostics functions. We provide sample code in $\texttt{Python}$ and $\texttt{R}$ as well as examples of applications to photometric redshift estimation and likelihood-free cosmology via CDE.

Read this paper on arXiv…

N. Dalmasso, T. Pospisil, A. Lee, et. al.
Mon, 2 Sep 19
47/66

Comments: 23 pages, 4 figures, 3 tables

Bayesian automated posterior repartitioning for nested sampling [CL]

http://arxiv.org/abs/1908.04655


Priors in Bayesian analyses often encode informative domain knowledge that can be useful in making the inference process more efficient. Occasionally, however, priors may be unrepresentative of the parameter values for a given dataset, which can result in inefficient parameter space exploration, or even incorrect inferences, particularly for nested sampling (NS) algorithms. Simply broadening the prior in such cases may be inappropriate or impossible in some applications. Hence a previous solution of this problem, known as posterior repartitioning (PR), redefines the prior and likelihood while keeping their product fixed, so that the posterior inferences and evidence estimates remain unchanged, but the efficiency of the NS process is significantly increased. In its most practical form, PR raises the prior to some power $\beta$, which is introduced as an auxiliary variable that must be determined on a case-by-case basis, usually by lowering $\beta$ from unity according to some pre-defined annealing schedule' until the resulting inferences converge to a consistent solution. We present here an alternative Bayesianautomated PR’ method, in which $\beta$ is instead treated as a hyperparameter that is inferred from the data alongside the original parameters of the problem, and then marginalised over to obtain the final inference. We show through numerical examples that this approach provides a robust and efficient `hands-off’ solution to addressing the issue of unrepresentative priors in Bayesian inference using NS. Moreover, we show that for problems with representative priors the method has a negligible computational overhead relative to standard nesting sampling, which suggests that it should be used in as a matter of course in all NS analyses.

Read this paper on arXiv…

X. Chen, F. Feroz and M. Hobson
Wed, 14 Aug 19
18/60

Comments: N/A

rcosmo: R Package for Analysis of Spherical, HEALPix and Cosmological Data [CL]

http://arxiv.org/abs/1907.05648


The analysis of spatial observations on a sphere is important in areas such as geosciences, physics and embryo research, just to name a few. The purpose of the package rcosmo is to conduct efficient information processing, visualisation, manipulation and spatial statistical analysis of Cosmic Microwave Background (CMB) radiation and other spherical data. The package was developed for spherical data stored in the Hierarchical Equal Area isoLatitude Pixelation (Healpix) representation. rcosmo has more than 100 different functions. Most of them initially were developed for CMB, but also can be used for other spherical data as rcosmo contains tools for transforming spherical data in cartesian and geographic coordinates into the HEALPix representation. We give a general description of the package and illustrate some important functionalities and benchmarks.

Read this paper on arXiv…

D. Fryer, M. Li and A. Olenko
Mon, 15 Jul 19
25/38

Comments: 20 pages, 15 figures

Gaussbock: Fast parallel-iterative cosmological parameter estimation with Bayesian nonparametrics [CEA]

http://arxiv.org/abs/1905.09800


We present and apply Gaussbock, a new embarrassingly parallel iterative algorithm for cosmological parameter estimation designed for an era of cheap parallel computing resources. Gaussbock uses Bayesian nonparametrics and truncated importance sampling to accurately draw samples from posterior distributions with an orders-of-magnitude speed-up in wall time over alternative methods. Contemporary problems in this area often suffer from both increased computational costs due to high-dimensional parameter spaces and consequent excessive time requirements, as well as the need for fine tuning of proposal distributions or sampling parameters. Gaussbock is designed specifically with these issues in mind. We explore and validate the performance and convergence of the algorithm on a fast approximation to the Dark Energy Survey Year 1 (DES Y1) posterior, finding reasonable scaling behavior with the number of parameters. We then test on the full DES Y1 posterior using large-scale supercomputing facilities, and recover reasonable agreement with previous chains, although the algorithm can underestimate the tails of poorly-constrained parameters. In addition, we provide the community with a user-friendly software tool for accelerated cosmological parameter estimation based on the methodology described in this paper.

Read this paper on arXiv…

B. Moews and J. Zuntz
Fri, 24 May 19
58/60

Comments: 17 pages, 7 figures, preprint to be submitted to ApJ

Nested sampling on non-trivial geometries [CL]

http://arxiv.org/abs/1905.09110


Metropolis nested sampling evolves a Markov chain from a current livepoint and accepts new points along the chain according to a version of the Metropolis acceptance ratio modified to satisfy the likelihood constraint, characteristic of nested sampling algorithms. The geometric nested sampling algorithm we present here is a based on the Metropolis method, but treats parameters as though they represent points on certain geometric objects, namely circles, tori and spheres. For parameters which represent points on a circle or torus, the trial distribution is `wrapped’ around the domain of the posterior distribution such that samples cannot be rejected automatically when evaluating the Metropolis ratio due to being outside the sampling domain. Furthermore, this enhances the mobility of the sampler. For parameters which represent coordinates on the surface of a sphere, the algorithm transforms the parameters into a Cartesian coordinate system before sampling which again makes sure no samples are automatically rejected, and provides a physically intutive way of the sampling the parameter space. \ We apply the geometric nested sampler to two types of toy model which include circular, toroidal and spherical parameters. We find that the geometric nested sampler generally outperforms \textsc{MultiNest} in both cases. \ %We also apply the algorithm to a gravitational wave detection model which includes circular and spherical parameters, and find that the geometric nested sampler and \textsc{MultiNest} appear to perform equally well as one another. Our implementation of the algorithm can be found at \url{https://github.com/SuperKam91/nested_sampling}.

Read this paper on arXiv…

K. Javid
Thu, 23 May 19
21/67

Comments: 13 pages, 11 figures, 28 equations

dynesty: A Dynamic Nested Sampling Package for Estimating Bayesian Posteriors and Evidences [IMA]

http://arxiv.org/abs/1904.02180


We present dynesty, a public, open-source, Python package to estimate Bayesian posteriors and evidences (marginal likelihoods) using Dynamic Nested Sampling. By adaptively allocating samples based on posterior structure, Dynamic Nested Sampling has the benefits of Markov Chain Monte Carlo algorithms that focus exclusively on posterior estimation while retaining Nested Sampling’s ability to estimate evidences and sample from complex, multi-modal distributions. We provide an overview of Nested Sampling, its extension to Dynamic Nested Sampling, the algorithmic challenges involved, and the various approaches taken to solve them. We then examine dynesty’s performance on a variety of toy problems along with several astronomical applications. We find in particular problems dynesty can provide substantial improvements in sampling efficiency compared to popular MCMC approaches in the astronomical literature. More detailed statistical results related to Nested Sampling are also included in the Appendix.

Read this paper on arXiv…

J. Speagle
Fri, 5 Apr 19
54/55

Comments: 28 pages, 12 figures, submitted to MNRAS; code available at this https URL

Posterior inference unchained with EL_2O [CL]

http://arxiv.org/abs/1901.04454


Statistical inference of analytically non-tractable posteriors is a difficult problem because of marginalization of correlated variables and stochastic methods such as MCMC and VI are commonly used. We argue that stochastic KL divergence minimization used by MCMC and VI is noisy, and we propose instead EL_2O, expectation optimization of L_2 distance squared between the approximate log posterior q and the un-normalized log posterior of p. When sampling from q the solutions agree with stochastic KL divergence minimization based VI in the large sample limit, however EL_2O method is free of sampling noise, has better optimization properties, and requires only as many sample evaluations as the number of parameters we are optimizing if q covers p. As a consequence, increasing the expressivity of q improves both the quality of results and the convergence rate, allowing EL_2O to approach exact inference. Use of automatic differentiation methods enables us to develop Hessian, gradient and gradient free versions of the method, which can determine M(M+2)/2+1, M+1 and 1 parameter(s) of q with a single sample, respectively. EL_2O provides a reliable estimate of the quality of the approximating posterior, and converges rapidly on full rank gaussian approximation for q and extensions beyond it, such as nonlinear transformations and gaussian mixtures. These can handle general posteriors, while still allowing fast analytic marginalizations. We test it on several examples, including a realistic 13 dimensional galaxy clustering analysis, showing that it is several orders of magnitude faster than MCMC, while giving smooth and accurate non-gaussian posteriors, often requiring a few to a few dozen of iterations only.

Read this paper on arXiv…

U. Seljak and B. Yu
Fri, 18 Jan 19
43/55

Comments: 35 pages, 6 figures

Stress testing the dark energy equation of state imprint on supernova data [CEA]

http://arxiv.org/abs/1812.09786


This work determines the degree to which a standard Lambda-CDM analysis based on type Ia supernovae can identify deviations from a cosmological constant in the form of a redshift-dependent dark energy equation of state w(z). We introduce and apply a novel random curve generator to simulate instances of w(z) from constraint families with increasing distinction from a cosmological constant. After producing a series of mock catalogs of binned type Ia supernovae corresponding to each w(z) curve, we perform a standard Lambda-CDM analysis to estimate the corresponding posterior densities of the absolute magnitude of type Ia supernovae, the present-day matter density, and the equation of state parameter. Using the Kullback-Leibler divergence between posterior densities as a difference measure, we demonstrate that a standard type Ia supernova cosmology analysis has limited sensitivity to extensive redshift dependencies of the dark energy equation of state. In addition, we report that larger redshift-dependent departures from a cosmological constant do not necessarily manifest easier-detectable incompatibilities with the Lambda-CDM model. Our results suggest that physics beyond the standard model may simply be hidden in plain sight.

Read this paper on arXiv…

B. Moews, R. Souza, E. Ishida, et. al.
Thu, 27 Dec 18
37/80

Comments: 13 pages, 9 figures, preprint submitted to PRD

Stress testing the dark energy equation of state imprint on supernova data [CEA]

http://arxiv.org/abs/1812.09786


This work determines the degree to which a standard Lambda-CDM analysis based on type Ia supernovae can identify deviations from a cosmological constant in the form of a redshift-dependent dark energy equation of state w(z). We introduce and apply a novel random curve generator to simulate instances of w(z) from constraint families with increasing distinction from a cosmological constant. After producing a series of mock catalogs of binned type Ia supernovae corresponding to each w(z) curve, we perform a standard Lambda-CDM analysis to estimate the corresponding posterior densities of the absolute magnitude of type Ia supernovae, the present-day matter density, and the equation of state parameter. Using the Kullback-Leibler divergence between posterior densities as a difference measure, we demonstrate that a standard type Ia supernova cosmology analysis has limited sensitivity to extensive redshift dependencies of the dark energy equation of state. In addition, we report that larger redshift-dependent departures from a cosmological constant do not necessarily manifest easier-detectable incompatibilities with the Lambda-CDM model. Our results suggest that physics beyond the standard model may simply be hidden in plain sight.

Read this paper on arXiv…

B. Moews, R. Souza, E. Ishida, et. al.
Thu, 27 Dec 18
24/80

Comments: 13 pages, 9 figures, preprint submitted to PRD

Compressed sensing and Sequential Monte Carlo for solar hard X-ray imaging [SSA]

http://arxiv.org/abs/1812.08413


We describe two inversion methods for the reconstruction of hard X-ray solar images. The methods are tested against experimental visibilities recorded by the Reuven Ramaty High Energy Solar Spectroscopic Imager (RHESSI) and synthetic visibilities based on the design of the Spectrometer/Telescope for Imaging X-rays (STIX).

Read this paper on arXiv…

A. Massone, F. Sciacchitano, M. Piana, et. al.
Fri, 21 Dec 18
45/72

Comments: submitted to ‘Nuovo Cimento’ as proceeding SOHE3

Sampling from manifold-restricted distributions using tangent bundle projections [CL]

http://arxiv.org/abs/1811.05494


A common problem in Bayesian inference is the sampling of target probability distributions at sufficient resolution and accuracy to estimate the probability density, and to compute credible regions. Often by construction, many target distributions can be expressed as some higher-dimensional closed-form distribution with parametrically constrained variables; i.e. one that is restricted to a smooth submanifold of Euclidean space. I propose a derivative-based importance sampling framework for such distributions. A base set of $n$ samples from the target distribution is used to map out the tangent bundle of the manifold, and to seed $nm$ additional points that are projected onto the tangent bundle and weighted appropriately. The method can act as a multiplicative complement to any standard sampling algorithm, and is designed for the efficient production of approximate high-resolution histograms from manifold-restricted Gaussian distributions.

Read this paper on arXiv…

A. Chua
Thu, 15 Nov 18
43/56

Comments: 28 pages, 6 figures

Improving the efficiency and robustness of nested sampling using posterior repartitioning [CL]

http://arxiv.org/abs/1803.06387


In real-world Bayesian inference applications, prior assumptions regarding the parameters of interest may be unrepresentative of their actual values for a given dataset. In particular, if the likelihood is concentrated far out in the wings of the assumed prior distribution, this can lead to extremely inefficient exploration of the resulting posterior by nested sampling algorithms, with unnecessarily high associated computational costs. Simple solutions such as broadening the prior range in such cases might not be appropriate or possible in real-world applications, for example when one wishes to assume a single standardised prior across the analysis of a large number of datasets for which the true values of the parameters of interest may vary. This work therefore introduces a posterior repartitioning (PR) method for nested sampling algorithms, which addresses the problem by redefining the likelihood and prior while keeping their product fixed, so that the posterior inferences and evidence estimates remain unchanged but the efficiency of the nested sampling process is significantly increased. Numerical results show that the PR method provides a simple yet powerful refinement for nested sampling algorithms to address the issue of unrepresentative priors.

Read this paper on arXiv…

X. Chen, M. Hobson, S. Das, et. al.
Thu, 1 Nov 18
32/76

Comments: N/A

Bayesian cosmic density field inference from redshift space dark matter maps [CEA]

http://arxiv.org/abs/1810.05189


We present a self-consistent Bayesian formalism to sample the primordial density fields compatible with a set of dark matter density tracers after cosmic evolution observed in redshift space. Previous works on density reconstruction considered redshift space distortions as noise or included an additional iterative distortion correction step. We present here the analytic solution of coherent flows within a Hamiltonian Monte Carlo posterior sampling of the primordial density field. We test our method within the Zel’dovich approximation, presenting also an analytic solution including tidal fields and spherical collapse on small scales using augmented Lagrangian perturbation theory. Our resulting reconstructed fields are isotropic and their power spectra are unbiased compared to the true one defined by our mock observations. Novel algorithmic implementations are introduced regarding the mass assignment kernels when defining the dark matter density field and optimization of the time step in the Hamiltonian equations of motions. Our algorithm, dubbed barcode, promises to be especially suited for analysis of the dark matter cosmic web implied by the observed spatial distribution of galaxy clusters — such as obtained from X-ray, SZ or weak lensing surveys — as well as that of the intergalactic medium sampled by the Lyman alpha forest or perhaps even by deep hydrogen intensity mapping. In these cases, virialized motions are negligible, and the tracers cannot be modeled as point-like objects. It could be used in all of these contexts as a baryon acoustic oscillation reconstruction algorithm.

Read this paper on arXiv…

E. Bos, F. Kitaura and R. Weygaert
Mon, 15 Oct 18
41/56

Comments: 33 pages, 20 figures, 1 table. Submitted to MNRAS. Accompanying code at this https URL

Diagnostic Tests for Nested Sampling Calculations [CL]

http://arxiv.org/abs/1804.06406


Nested sampling is an increasingly popular technique for Bayesian computation – in particular for multimodal, degenerate and high-dimensional problems. Without appropriate settings, however, nested sampling software may fail to explore such posteriors fully; for example producing correlated samples or missing significant modes. This paper introduces new diagnostic tests to assess the reliability of both parameter estimation and evidence calculations using nested sampling software, and demonstrates them empirically. We present two new diagnostic plots for nested sampling, and give practical advice for nested sampling software users. Our diagnostic tests and diagrams are implemented in nestcheck: a publicly available python package for analysing nested sampling calculations which is compatible with results from MultiNest and PolyChord.

Read this paper on arXiv…

E. Higson, W. Handley, M. Hobson, et. al.
Thu, 19 Apr 18
39/47

Comments: 21 pages + appendix, 13 figures

Constrained Least Squares for Extended Complex Factor Analysis [CL]

http://arxiv.org/abs/1804.00430


For subspace estimation with an unknown colored noise, Factor Analysis (FA) is a good candidate for replacing the popular eigenvalue decomposition (EVD). Finding the unknowns in factor analysis can be done by solving a non-linear least square problem. For this type of optimization problems, the Gauss-Newton (GN) algorithm is a powerful and simple method. The most expensive part of the GN algorithm is finding the direction of descent by solving a system of equations at each iteration. In this paper we show that for FA, the matrices involved in solving these systems of equations can be diagonalized in a closed form fashion and the solution can be found in a computationally efficient way. We show how the unknown parameters can be updated without actually constructing these matrices. The convergence performance of the algorithm is studied via numerical simulations.

Read this paper on arXiv…

A. Sardarabadi, A. Veen and L. Koopmans
Tue, 3 Apr 18
48/57

Comments: N/A

Testing One Hypothesis Multiple Times: The Multidimensional Case [CL]

http://arxiv.org/abs/1803.03858


The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection of competing models, are among the most challenging problems in statistical practice. These challenges can be tackled using a test of hypothesis where a nuisance parameter is present only under the alternative, and a computationally efficient solution can be obtained by the “Testing One Hypothesis Multiple times” (TOHM) method. In the one-dimensional setting, a fine discretization of the space of the non-identifiable parameter is specified, and a global p-value is obtained by approximating the distribution of the supremum of the resulting stochastic process. In this paper, we propose a computationally efficient inferential tool to perform TOHM in the multidimensional setting. Here, the approximations of interest typically involve the expected Euler Characteristics (EC) of the excursion set of the underlying random field. We introduce a simple algorithm to compute the EC in multiple dimensions and for arbitrary large significance levels. This leads to an highly generalizable computational tool to perform inference under non-standard regularity conditions.

Read this paper on arXiv…

S. Algeri and D. van Dyk
Tue, 13 Mar 2018
36/61

Comments: N/A

An efficient $k$-means-type algorithm for clustering datasets with incomplete records [CL]

http://arxiv.org/abs/1802.08363


The $k$-means algorithm is the most popular nonparametric clustering method in use, but cannot generally be applied to data sets with missing observations. The usual practice with such data sets is to either impute the values under an assumption of a missing-at-random mechanism or to ignore the incomplete records, and then to use the desired clustering method. We develop an efficient version of the $k$-means algorithm that allows for clustering cases where not all the features have observations recorded. Our extension is called $k_m$-means and reduces to the $k$-means algorithm when all records are complete. We also provide strategies to initialize our algorithm and to estimate the number of groups in the data set. Illustrations and simulations demonstrate the efficacy of our approach in a variety of settings and patterns of missing data. Our methods are also applied to the clustering of gamma-ray bursts and to the analysis of activation images obtained from a functional Magnetic Resonance Imaging experiment.

Read this paper on arXiv…

A. Lithio and R. Maitra
Mon, 26 Feb 18
36/49

Comments: 23 pages, 14 figures, 2 tables

Identification of multiple hard X-ray sources in solar flares: A Bayesian analysis of the February 20 2002 event [SSA]

http://arxiv.org/abs/1801.09141


Hard X-ray emission in solar flares is typically characterized by a number of discrete sources, each with its own spectral, temporal, and spatial variability. Establishing the relationship amongst these sources is critical to determine the role of each in the energy release and transport processes that occur within the flare. In this paper we present a novel method to identify and characterize each source of hard X-ray emission. In particular, the method permits a quantitative determination of the most likely number of subsources present, and of the relative probabilities that the hard X-ray emission in a given subregion of the flare is represented by a complicated multiple source structure or by a simpler single source. We apply the method to a well-studied flare on 2002~February~20 in order to assess competing claims as to the number of chromospheric footpoint sources present, and hence to the complexity of the underlying magnetic geometry/toplogy. Contrary to previous claims of the need for multiple sources to account for the chromospheric hard X-ray emission at different locations and times, we find that a simple two-footpoint-plus-coronal-source model is the most probable explanation for the data. We also find that one of the footpoint sources moves quite rapidly throughout the event, a factor that presumably complicated previous analyses. The inferred velocity of the footpoint corresponds to a very high induced electric field, compatible with those in thin reconnecting current sheets.

Read this paper on arXiv…

F. Scacchiano, A. Sorrentino, A. Emslie, et. al.
Tue, 30 Jan 18
19/70

Comments: submitted to ApJ

Data analysis recipes: Using Markov Chain Monte Carlo [IMA]

http://arxiv.org/abs/1710.06068


Markov Chain Monte Carlo (MCMC) methods for sampling probability density functions (combined with abundant computational resources) have transformed the sciences, especially in performing probabilistic inferences, or fitting models to data. In this primarily pedagogical contribution, we give a brief overview of the most basic MCMC method and some practical advice for the use of MCMC in real inference problems. We give advice on method choice, tuning for performance, methods for initialization, tests of convergence, troubleshooting, and use of the chain output to produce or report parameter estimates with associated uncertainties. We argue that autocorrelation time is the most important test for convergence, as it directly connects to the uncertainty on the sampling estimate of any quantity of interest. We emphasize that sampling is a method for doing integrals; this guides our thinking about how MCMC output is best used.

Read this paper on arXiv…

D. Hogg and D. Foreman-Mackey
Wed, 18 Oct 2017
27/62

Comments: A purely pedagogical contribution

Big Data vs. complex physical models: a scalable inference algorithm [CL]

http://arxiv.org/abs/1707.04476


The data torrent unleashed by current and upcoming instruments requires scalable analysis methods. Machine Learning approaches scale well. However, separating the instrument measurement from the physical effects of interest, dealing with variable errors, and deriving parameter uncertainties is usually an after-thought. Classic forward-folding analyses with Markov Chain Monte Carlo or Nested Sampling enable parameter estimation and model comparison, even for complex and slow-to-evaluate physical models. However, these approaches require independent runs for each data set, implying an unfeasible number of model evaluations in the Big Data regime. Here we present a new algorithm based on nested sampling, deriving parameter probability distributions for each observation. Importantly, in our method the number of physical model evaluations scales sub-linearly with the number of data sets, and we make no assumptions about homogeneous errors, Gaussianity, the form of the model or heterogeneity/completeness of the observations. Our method has immediate application in speeding up analyses of large surveys, integral-field-unit observations, and Monte Carlo simulations.

Read this paper on arXiv…

J. Buchner
Mon, 17 Jul 17
44/45

Comments: Submitted to MNRAS. Comments welcome. Figure 6 demonstrates the scaling. Implementation at this https URL

Computing Entropies With Nested Sampling [CL]

http://arxiv.org/abs/1707.03543


The Shannon entropy, and related quantities such as mutual information, can be used to quantify uncertainty and relevance. However, in practice, it can be difficult to compute these quantities for arbitrary probability distributions, particularly if the probability mass functions or densities cannot be evaluated. This paper introduces a computational approach, based on Nested Sampling, to evaluate entropies of probability distributions that can only be sampled. I demonstrate the method on three examples: a simple gaussian example where the key quantities are available analytically; (ii) an experimental design example about scheduling observations in order to measure the period of an oscillating signal; and (iii) predicting the future from the past in a heavy-tailed scenario.

Read this paper on arXiv…

B. Brewer
Thu, 13 Jul 17
43/60

Comments: Submitted to Entropy. 18 pages, 3 figures. Software available at this https URL

Markov Chain Monte Carlo Methods for Bayesian Data Analysis in Astronomy [IMA]

http://arxiv.org/abs/1706.01629


Markov Chain Monte Carlo based Bayesian data analysis has now become the method of choice for analyzing and interpreting data in almost all disciplines of science. In astronomy, over the last decade, we have also seen a steady increase in the number of papers that employ Monte Carlo based Bayesian analysis. New, efficient Monte Carlo based methods are continuously being developed and explored. In this review, we first explain the basics of Bayesian theory and discuss how to set up data analysis problems within this framework. Next, we provide an overview of various Monte Carlo based methods for performing Bayesian data analysis. Finally, we discuss advanced ideas that enable us to tackle complex problems and thus hold great promise for the future. We also distribute downloadable computer software (available at https://github.com/sanjibs/bmcmc/ ) that implements some of the algorithms and examples discussed here.

Read this paper on arXiv…

S. Sharma
Wed, 7 Jun 17
7/52

Comments: 49 pages, draft version, to appear in Annual Review of Astronomy and Astrophysics

Comparison of statistical sampling methods with ScannerBit, the GAMBIT scanning module [CL]

http://arxiv.org/abs/1705.07959


We introduce ScannerBit, the statistics and sampling module of the public, open-source global fitting framework Gambit. ScannerBit provides a standardised interface to different sampling algorithms, enabling the use and comparison of multiple computational methods for inferring profile likelihoods, Bayesian posteriors, and other statistical quantities. The current version offers random, grid, raster, nested sampling, differential evolution, Markov Chain Monte Carlo (MCMC) and ensemble Monte Carlo samplers. We also announce the release of a new standalone differential evolution sampler, Diver, and describe its design, usage and interface to ScannerBit. We subject Diver and three other samplers (the nested sampler MultiNest, the MCMC GreAT, and the native ScannerBit implementation of the ensemble Monte Carlo algorithm TWalk) to a battery of statistical tests. For this we use a realistic physical likelihood function, based on the scalar singlet model of dark matter. We examine the performance of each sampler as a function of its adjustable settings, and the dimensionality of the sampling problem. We evaluate performance on four metrics: optimality of the best fit found, completeness in exploring the best-fit region, number of likelihood evaluations, and total runtime. For Bayesian posterior estimation at high resolution, TWalk provides the most accurate and timely mapping of the full parameter space. For profile likelihood analysis in less than about ten dimensions, we find that Diver and MultiNest score similarly in terms of best fit and speed, outperforming GreAT and TWalk; in ten or more dimensions, Diver substantially outperforms the other three samplers on all metrics.

Read this paper on arXiv…

G. Workgroup, G. Martinez, J. McKay, et. al.
Wed, 24 May 17
14/70

Comments: 46 pages, 18 figures, 2 tables, submitted to EPJC

Dynamic nested sampling: an improved algorithm for parameter estimation and evidence calculation [CL]

http://arxiv.org/abs/1704.03459


We introduce dynamic nested sampling: a generalisation of the nested sampling algorithm in which the number of “live points” varies to allocate samples more efficiently. In empirical tests the new method increases accuracy by up to a factor of ~8 for parameter estimation and ~3 for evidence calculation compared to standard nested sampling with the same number of samples – equivalent to speeding up the computation by factors of ~64 and ~9 respectively. In addition unlike in standard nested sampling more accurate results can be obtained by continuing the calculation for longer. Dynamic nested sampling can be easily included in existing nested sampling software such as MultiNest and PolyChord.

Read this paper on arXiv…

E. Higson, W. Handley, M. Hobson, et. al.
Thu, 13 Apr 17
36/56

Comments: 16 pages + appendix, 8 figures, submitted to Bayesian Analysis. arXiv admin note: text overlap with arXiv:1703.09701

Marginal Likelihoods from Monte Carlo Markov Chains [CL]

http://arxiv.org/abs/1704.03472


In this paper, we present a method for computing the marginal likelihood, also known as the model likelihood or Bayesian evidence, from Markov Chain Monte Carlo (MCMC), or other sampled posterior distributions. In order to do this, one needs to be able to estimate the density of points in parameter space, and this can be challenging in high numbers of dimensions. Here we present a Bayesian analysis, where we obtain the posterior for the marginal likelihood, using $k$th nearest-neighbour distances in parameter space, using the Mahalanobis distance metric, under the assumption that the points in the chain (thinned if required) are independent. We generalise the algorithm to apply to importance-sampled chains, where each point is assigned a weight. We illustrate this with an idealised posterior of known form with an analytic marginal likelihood, and show that for chains of length $\sim 10^5$ points, the technique is effective for parameter spaces with up to $\sim 20$ dimensions. We also argue that $k=1$ is the optimal choice, and discuss failure modes for the algorithm. In a companion paper (Heavens et al. 2017) we apply the technique to the main MCMC chains from the 2015 Planck analysis of cosmic background radiation data, to infer that quantitatively the simplest 6-parameter flat $\Lambda$CDM standard model of cosmology is preferred over all extensions considered.

Read this paper on arXiv…

A. Heavens, Y. Fantaye, A. Mootoovaloo, et. al.
Thu, 13 Apr 17
51/56

Comments: N/A

DNest4: Diffusive Nested Sampling in C++ and Python [CL]

http://arxiv.org/abs/1606.03757


In probabilistic (Bayesian) inferences, we typically want to compute properties of the posterior distribution, describing knowledge of unknown quantities in the context of a particular dataset and the assumed prior information. The marginal likelihood, also known as the “evidence”, is a key quantity in Bayesian model selection. The Diffusive Nested Sampling algorithm, a variant of Nested Sampling, is a powerful tool for generating posterior samples and estimating marginal likelihoods. It is effective at solving complex problems including many where the posterior distribution is multimodal or has strong dependencies between variables. DNest4 is an open source (MIT licensed), multi-threaded implementation of this algorithm in C++11, along with associated utilities including: i) RJObject, a class template for finite mixture models, (ii) A Python package allowing basic use without C++ coding, and iii) Experimental support for models implemented in Julia. In this paper we demonstrate DNest4 usage through examples including simple Bayesian data analysis, finite mixture models, and Approximate Bayesian Computation.

Read this paper on arXiv…

B. Brewer and D. Foreman-Mackey
Tue, 14 Jun 16
40/67

Comments: Submitted. 31 pages, 9 figures

Numerical methods for solution of the stochastic differential equations equivalent to the non-stationary Parker's transport equation [SSA]

http://arxiv.org/abs/1509.06890


We derive the numerical schemes for the strong order integration of the set of the stochastic differential equations (SDEs) corresponding to the non-stationary Parker transport equation (PTE). PTE is 5-dimensional (3 spatial coordinates, particles energy and time) Fokker- Planck type equation describing the non-stationary the galactic cosmic ray (GCR) particles transport in the heliosphere. We present the formulas for the numerical solution of the obtained set of SDEs driven by a Wiener process in the case of the full three-dimensional diffusion tensor. We introduce the solution applying the strong order Euler-Maruyama, Milstein and stochastic Runge-Kutta methods. We discuss the advantages and disadvantages of the presented numerical methods in the context of increasing the accuracy of the solution of the PTE.

Read this paper on arXiv…

A. Wawrzynczak, R. Modzelewska and M. Kluczek
Thu, 24 Sep 15
20/60

Comments: 4 pages, 2 figures, presented on 4th International Conference on Mathematical Modeling in Physical Sciences, 2015

Stochastic approach to the numerical solution of the non-stationary Parker's transport equation [SSA]

http://arxiv.org/abs/1509.06523


We present the newly developed stochastic model of the galactic cosmic ray (GCR) particles transport in the heliosphere. Mathematically Parker transport equation (PTE) describing non-stationary transport of charged particles in the turbulent medium is the Fokker-Planck type. It is the second order parabolic time-dependent 4-dimensional (3 spatial coordinates and particles energy/rigidity) partial differential equation. It is worth to mention that, if we assume the stationary case it remains as the 3-D parabolic type problem with respect to the particles rigidity R. If we fix the energy it still remains as the 3-D parabolic type problem with respect to time. The proposed method of numerical solution is based on the solution of the system of stochastic differential equations (SDEs) being equivalent to the Parker’s transport equation. We present the method of deriving from PTE the equivalent SDEs in the heliocentric spherical coordinate system for the backward approach. The obtained stochastic model of the Forbush decrease of the GCR intensity is in an agreement with the experimental data. The advantages and disadvantages of the forward and the backward solution of the PTE are discussed.

Read this paper on arXiv…

A. Wawrzynczak, R. Modzelewska and A. Gil
Wed, 23 Sep 15
38/63

Comments: 4 pages, 2 figures, presented on International Conference on Mathematical Modeling in Physical Sciences, 2014

A stochastic method of solution of the Parker transport equation [SSA]

http://arxiv.org/abs/1509.06519


We present the stochastic model of the galactic cosmic ray (GCR) particles transport in the heliosphere. Based on the solution of the Parker transport equation we developed models of the short-time variation of the GCR intensity, i.e. the Forbush decrease (Fd) and the 27-day variation of the GCR intensity. Parker transport equation being the Fokker-Planck type equation delineates non-stationary transport of charged particles in the turbulent medium. The presented approach of the numerical solution is grounded on solving of the set of equivalent stochastic differential equations (SDEs). We demonstrate the method of deriving from Parker transport equation the corresponding SDEs in the heliocentric spherical coordinate system for the backward approach. Features indicative the preeminence of the backward approach over the forward is stressed. We compare the outcomes of the stochastic model of the Fd and 27-day variation of the GCR intensity with our former models established by the finite difference method. Both models are in an agreement with the experimental data.

Read this paper on arXiv…

A. Wawrzynczak, R. Modzelewska and A. Gil
Wed, 23 Sep 15
46/63

Comments: 8 pages, 7 figures, presented on 24th European Cosmic Ray Symposium 2014

Uncertainty for calculating transport on Titan: a probabilistic description of bimolecular diffusion parameters [EPA]

http://arxiv.org/abs/1508.02818


Bimolecular diffusion coefficients are important parameters used by atmospheric models to calculate altitude profiles of minor constituents in an atmosphere. Unfortunately, laboratory measurements of these coefficients were never conducted at temperature conditions relevant to the atmosphere of Titan. Here we conduct a detailed uncertainty analysis of the bimolecular diffusion coefficient parameters as applied to Titan’s upper atmosphere to provide a better understanding of the impact of uncertainty for this parameter on models. Because temperature and pressure conditions are much lower than the laboratory conditions in which bimolecular diffusion parameters were measured, we apply a Bayesian framework, a problem-agnostic framework, to determine parameter estimates and associated uncertainties. We solve the Bayesian calibration problem using the open-source QUESO library which also performs a propagation of uncertainties in the calibrated parameters to temperature and pressure conditions observed in Titan’s upper atmosphere. Our results show that, after propagating uncertainty through the Massman model, the uncertainty in molecular diffusion is highly correlated to temperature and we observe no noticeable correlation with pressure. We propagate the calibrated molecular diffusion estimate and associated uncertainty to obtain an estimate with uncertainty due to bimolecular diffusion for the methane molar fraction as a function of altitude. Results show that the uncertainty in methane abundance due to molecular diffusion is in general small compared to eddy diffusion and the chemical kinetics description. However, methane abundance is most sensitive to uncertainty in molecular diffusion above 1200 km where the errors are nontrivial and could have important implications for scientific research based on diffusion models in this altitude range.

Read this paper on arXiv…

S. Plessis, D. McDougall, K. Mandt, et. al.
Thu, 13 Aug 15
44/49

Comments: N/A

Approximate Bayesian Computation for Forward Modeling in Cosmology [CEA]

http://arxiv.org/abs/1504.07245


Bayesian inference is often used in cosmology and astrophysics to derive constraints on model parameters from observations. This approach relies on the ability to compute the likelihood of the data given a choice of model parameters. In many practical situations, the likelihood function may however be unavailable or intractable due to non-gaussian errors, non-linear measurements processes, or complex data formats such as catalogs and maps. In these cases, the simulation of mock data sets can often be made through forward modeling. We discuss how Approximate Bayesian Computation (ABC) can be used in these cases to derive an approximation to the posterior constraints using simulated data sets. This technique relies on the sampling of the parameter set, a distance metric to quantify the difference between the observation and the simulations and summary statistics to compress the information in the data. We first review the principles of ABC and discuss its implementation using a Population Monte-Carlo (PMC) algorithm. We test the performance of the implementation using a Gaussian toy model. We then apply the ABC technique to the practical case of the calibration of image simulations for wide field cosmological surveys. We find that the ABC analysis is able to provide reliable parameter constraints for this problem and is therefore a promising technique for other applications in cosmology and astrophysics. Our implementation of the ABC PMC method is made available via a public code release.

Read this paper on arXiv…

J. Akeret, A. Refregier, A. Amara, et. al.
Wed, 29 Apr 15
3/62

Comments: Submitted to Journal of Cosmology and Astroparticle Physics. 16 pages, 5 figures, 1 algorithm. The code is available at this https URL

Stochastic determination of matrix determinants [CL]

http://arxiv.org/abs/1504.02661


Matrix determinants play an important role in data analysis, in particular when Gaussian processes are involved. Due to currently exploding data volumes linear operations – matrices – acting on the data are often not accessible directly, but are only represented indirectly in form of a computer routine. Such a routine implements the transformation a data vector undergoes under matrix multiplication. Meanwhile efficient probing routines to estimate a matrix’s diagonal or trace, based solely on such computationally affordable matrix-vector multiplications, are well known and frequently used in signal inference, a stochastic estimate for its determinant is still lacking. In this work a probing method for the logarithm of a determinant of a linear operator is introduced. This method rests upon a reformulation of the log-determinant by an integral representation and the transformation of the involved terms into stochastic expressions. This stochastic determinant determination enables large-size applications in Bayesian inference, in particular evidence calculations, model comparison, and posterior determination.

Read this paper on arXiv…

S. Dorn and T. Ensslin
Mon, 13 Apr 15
49/54

Comments: 8 pages, 5 figures

Inference for Trans-dimensional Bayesian Models with Diffusive Nested Sampling [CL]

http://arxiv.org/abs/1411.3921


Many inference problems involve inferring the number $N$ of objects in some region, along with their properties $\{\mathbf{x}_i\}_{i=1}^N$, from a dataset $\mathcal{D}$. A common statistical example is finite mixture modelling. In the Bayesian framework, these problems are typically solved using one of the following two methods: i) by executing a Monte Carlo algorithm (such as Nested Sampling) once for each possible value of $N$, and calculating the marginal likelihood or evidence as a function of $N$; or ii) by doing a single run that allows the model dimension $N$ to change (such as Markov Chain Monte Carlo with birth/death moves), and obtaining the posterior for $N$ directly. In this paper we present a general approach to this problem that uses trans-dimensional MCMC embedded {\it within} a Nested Sampling algorithm, allowing us to explore the posterior distribution and calculate the marginal likelihood (summed over $N$) even if the problem contains a phase transition or other difficult features such as multimodality. We present two example problems, finding sinusoidal signals in noisy data, and finding and measuring galaxies in a noisy astronomical image. Both of the examples demonstrate phase transitions in the relationship between the likelihood and the cumulative prior mass.

Read this paper on arXiv…

B. Brewer
Mon, 17 Nov 14
1/52

Comments: Submitted. Comments welcome. 14 pages, 7 figures. Software available at this https URL

Bayesian Evidence and Model Selection [CL]

http://arxiv.org/abs/1411.3013


In this paper we review the concept of the Bayesian evidence and its application to model selection. The theory is presented along with a discussion of analytic, approximate and numerical techniques. Application to several practical examples within the context of signal processing are discussed.

Read this paper on arXiv…

K. Knuth, M. Habeck, N. Malakar, et. al.
Thu, 13 Nov 14
38/49

Comments: 39 pages, 8 figures. Submitted to DSP. Features theory, numerical methods and four applications

Efficient Exploration of Multi-Modal Posterior Distributions [IMA]

http://arxiv.org/abs/1408.3969


The Markov Chain Monte Carlo (MCMC) algorithm is a widely recognised as an efficient method for sampling a specified posterior distribution. However, when the posterior is multi-modal, conventional MCMC algorithms either tend to become stuck in one local mode, become non-Markovian or require an excessively long time to explore the global properties of the distribution. We propose a novel variant of MCMC, mixed MCMC, which exploits a specially designed proposal density to allow the generation candidate points from any of a number of different modes. This new method is efficient by design, and is strictly Markovian. We present our method and apply it to a toy model inference problem to demonstrate its validity.

Read this paper on arXiv…

Y. Hu, M. Hendry and I. Heng
Tue, 19 Aug 14
14/69

Comments: 6 pages, 1 figure

Estimating the distribution of Galaxy Morphologies on a continuous space [GA]

http://arxiv.org/abs/1406.7536


The incredible variety of galaxy shapes cannot be summarized by human defined discrete classes of shapes without causing a possibly large loss of information. Dictionary learning and sparse coding allow us to reduce the high dimensional space of shapes into a manageable low dimensional continuous vector space. Statistical inference can be done in the reduced space via probability distribution estimation and manifold estimation.

Read this paper on arXiv…

G. Vinci, P. Freeman, J. Newman, et. al.
Tue, 1 Jul 14
57/70

Comments: 4 pages, 3 figures, Statistical Challenges in 21st Century Cosmology, Proceedings IAU Symposium No. 306, 2014

Exploring Multi-Modal Distributions with Nested Sampling [IMA]

http://arxiv.org/abs/1312.5638


In performing a Bayesian analysis, two difficult problems often emerge. First, in estimating the parameters of some model for the data, the resulting posterior distribution may be multi-modal or exhibit pronounced (curving) degeneracies. Secondly, in selecting between a set of competing models, calculation of the Bayesian evidence for each model is computationally expensive using existing methods such as thermodynamic integration. Nested Sampling is a Monte Carlo method targeted at the efficient calculation of the evidence, but also produces posterior inferences as a by-product and therefore provides means to carry out parameter estimation as well as model selection. The main challenge in implementing Nested Sampling is to sample from a constrained probability distribution. One possible solution to this problem is provided by the Galilean Monte Carlo (GMC) algorithm. We show results of applying Nested Sampling with GMC to some problems which have proven very difficult for standard Markov Chain Monte Carlo (MCMC) and down-hill methods, due to the presence of large number of local minima and/or pronounced (curving) degeneracies between the parameters. We also discuss the use of Nested Sampling with GMC in Bayesian object detection problems, which are inherently multi-modal and require the evaluation of Bayesian evidence for distinguishing between true and spurious detections.

Read this paper on arXiv…

Fri, 20 Dec 13
40/56

D3PO – Denoising, Deconvolving, and Decomposing Photon Observations [IMA]

http://arxiv.org/abs/1311.1888


The analysis of astronomical images is a non-trivial task. The D3PO algorithm addresses the inference problem of denoising, deconvolving, and decomposing photon observations. The primary goal is the simultaneous reconstruction of the diffuse and point-like photon flux from a given photon count image. In order to discriminate between these morphologically different signal components, a probabilistic algorithm is derived in the language of information field theory based on a hierarchical Bayesian parameter model. The signal inference exploits prior information on the spatial correlation structure of the diffuse component and the brightness distribution of the spatially uncorrelated point-like sources. A maximum a posteriori solution and a solution minimizing the Gibbs free energy of the inference problem using variational Bayesian methods are discussed. Since the derivation of the solution does not dependent on the underlying position space, the implementation of the D3PO algorithm uses the NIFTY package to ensure operationality on various spatial grids and at any resolution. The fidelity of the algorithm is validated by the analysis of simulated data, including a realistic high energy photon count image showing a 32 x 32 arcmin^2 observation with a spatial resolution of 0.1 arcmin. In all tests the D3PO algorithm successfully denoised, deconvolved, and decomposed the data into a diffuse and a point-like signal estimate for the respective photon flux components.

Read this paper on arXiv…

Mon, 11 Nov 13
31/39