Tag Archives: Learning

SBAF: A New Activation Function for Artificial Neural Net based Habitability Classification [CL]

Posted on by

http://arxiv.org/abs/1806.01844


We explore the efficacy of using a novel activation function in Artificial Neural Networks (ANN) in characterizing exoplanets into different classes. We call this Saha-Bora Activation Function (SBAF) as the motivation is derived from long standing understanding of using advanced calculus in modeling habitability score of Exoplanets. The function is demonstrated to possess nice analytical properties and doesn’t seem to suffer from local oscillation problems. The manuscript presents the analytical properties of the activation function and the architecture implemented on the function. Keywords: Astroinformatics, Machine Learning, Exoplanets, ANN, Activation Function.

Read this paper on arXiv…

S. Saha, A. Mathur, K. Bora, et. al.
Thu, 7 Jun 18
42/51

Comments: arXiv admin note: substantial text overlap with arXiv:1805.08810

Using transfer learning to detect galaxy mergers [IMA]

Posted on by

http://arxiv.org/abs/1805.10289


We investigate the use of deep convolutional neural networks (deep CNNs) for automatic visual detection of galaxy mergers. Moreover, we investigate the use of transfer learning in conjunction with CNNs, by retraining networks first trained on pictures of everyday objects. We test the hypothesis that transfer learning is useful for improving classification performance for small training sets. This would make transfer learning useful for finding rare objects in astronomical imaging datasets. We find that these deep learning methods perform significantly better than current state-of-the-art merger detection methods based on nonparametric systems like CAS and GM$_{20}$. Our method is end-to-end and robust to image noise and distortions; it can be applied directly without image preprocessing. We also find that transfer learning can act as a regulariser in some cases, leading to better overall classification accuracy ($p = 0.02$). Transfer learning on our full training set leads to a lowered error rate from 0.038 $\pm$ 1 down to 0.032 $\pm$ 1, a relative improvement of 15%. Finally, we perform a basic sanity-check by creating a merger sample with our method, and comparing with an already existing, manually created merger catalogue in terms of colour-mass distribution and stellar mass function.

Read this paper on arXiv…

S. Ackermann, K. Schawinski, C. Zhang, et. al.
Tue, 29 May 18
2/73

Comments: Code and data on this https URL

Real-time regression analysis with deep convolutional neural networks [CL]

Posted on by

http://arxiv.org/abs/1805.02716


We discuss the development of novel deep learning algorithms to enable real-time regression analysis for time series data. We showcase the application of this new method with a timely case study, and then discuss the applicability of this approach to tackle similar challenges across science domains.

Read this paper on arXiv…

E. Huerta, D. George, Z. Zhao, et. al.
Wed, 9 May 18
19/55

Comments: 3 pages. Position Paper accepted to SciML2018: DOE ASCR Workshop on Scientific Machine Learning. North Bethesda, MD, United States, January 30-February 1, 2018

Local, algebraic simplifications of Gaussian random fields [CEA]

Posted on by

http://arxiv.org/abs/1805.03117


Many applications of Gaussian random fields and Gaussian random processes are limited by the computational complexity of evaluating the probability density function, which involves inverting the relevant covariance matrix. In this work, we show how that problem can be completely circumvented for the local Taylor coefficients of a Gaussian random field with a Gaussian (or square exponential') covariance function. Our results hold for any dimension of the field and to any order in the Taylor expansion. We present two applications. First, we show that this method can be used to explicitly generate non-trivial potential energy landscapes with many fields. This application is particularly useful when one is concerned with the field locally around special points (e.g.~maxima or minima), as we exemplify by the problem of cosmicmanyfield’ inflation in the early universe. Second, we show that this method has applications in machine learning, and greatly simplifies the regression problem of determining the hyperparameters of the covariance function given a training data set consisting of local Taylor coefficients at single point. An accompanying Mathematica notebook is available at https://doi.org/10.17863/CAM.22859.

Read this paper on arXiv…

T. Bjorkmo and M. Marsh
Wed, 9 May 18
49/55

Comments: 15 pages, 2 figures

Machine Learning in Astronomy: A Case Study in Quasar-Star Classification [IMA]

Posted on by

http://arxiv.org/abs/1804.05051


We present the results of various automated classification methods, based on machine learning (ML), of objects from data releases 6 and 7 (DR6 and DR7) of the Sloan Digital Sky Survey (SDSS), primarily distinguishing stars from quasars. We provide a careful scrutiny of approaches available in the literature and have highlighted the pitfalls in those approaches based on the nature of data used for the study. The aim is to investigate the appropriateness of the application of certain ML methods. The manuscript argues convincingly in favor of the efficacy of asymmetric AdaBoost to classify photometric data. The paper presents a critical review of existing study and puts forward an application of asymmetric AdaBoost, as an offspring of that exercise.

Read this paper on arXiv…

M. Viquar, S. Basak, A. Dasgupta, et. al.
Mon, 16 Apr 18
10/52

Comments: 10 pages, 8 figures

Classification of simulated radio signals using Wide Residual Networks for use in the search for extra-terrestrial intelligence [IMA]

Posted on by

http://arxiv.org/abs/1803.08624


We describe a new approach and algorithm for the detection of artificial signals and their classification in the search for extraterrestrial intelligence (SETI). The characteristics of radio signals observed during SETI research are often most apparent when those signals are represented as spectrograms. Additionally, many observed signals tend to share the same characteristics, allowing for sorting of the signals into different classes. For this work, complex-valued time-series data were simulated to produce a corpus of 140,000 signals from seven different signal classes. A wide residual neural network was then trained to classify these signal types using the gray-scale 2D spectrogram representation of those signals. An average $F_1$ score of 95.11\% was attained when tested on previously unobserved simulated signals. We also report on the performance of the model across a range of signal amplitudes.

Read this paper on arXiv…

G. Cox, S. Egly, G. Harp, et. al.
Mon, 26 Mar 18
36/43

Comments: 16 pages, 8 figures

Classification of simulated radio signals using Wide Residual Networks for use in the search for extra-terrestrial intelligence [IMA]

Posted on by

http://arxiv.org/abs/1803.08624


We describe a new approach and algorithm for the detection of artificial signals and their classification in the search for extraterrestrial intelligence (SETI). The characteristics of radio signals observed during SETI research are often most apparent when those signals are represented as spectrograms. Additionally, many observed signals tend to share the same characteristics, allowing for sorting of the signals into different classes. For this work, complex-valued time-series data were simulated to produce a corpus of 140,000 signals from seven different signal classes. A wide residual neural network was then trained to classify these signal types using the gray-scale 2D spectrogram representation of those signals. An average $F_1$ score of 95.11\% was attained when tested on previously unobserved simulated signals. We also report on the performance of the model across a range of signal amplitudes.

Read this paper on arXiv…

G. Cox, S. Egly, G. Harp, et. al.
Mon, 26 Mar 18
43/43

Comments: 16 pages, 8 figures

Approximate Inference for Constructing Astronomical Catalogs from Images [CL]

Posted on by

http://arxiv.org/abs/1803.00113


We present a new, fully generative model for constructing astronomical catalogs from optical telescope image sets. Each pixel intensity is treated as a Poisson random variable with a rate parameter that depends on the latent properties of stars and galaxies. These latent properties are themselves random, with scientific prior distributions constructed from large ancillary datasets. We compare two procedures for posterior inference: Markov chain Monte Carlo (MCMC) and variational inference (VI). MCMC excels at quantifying uncertainty while VI is 1000x faster. Both procedures outperform the current state-of-the-art method for measuring celestial bodies’ colors, shapes, and morphologies. On a supercomputer, the VI procedure efficiently uses 665,000 CPU cores (1.3 million hardware threads) to construct an astronomical catalog from 50 terabytes of images.

Read this paper on arXiv…

J. Regier, A. Miller, D. Schlegel, et. al.
Fri, 2 Mar 18
8/61

Comments: N/A

An efficient $k$-means-type algorithm for clustering datasets with incomplete records [CL]

Posted on by

http://arxiv.org/abs/1802.08363


The $k$-means algorithm is the most popular nonparametric clustering method in use, but cannot generally be applied to data sets with missing observations. The usual practice with such data sets is to either impute the values under an assumption of a missing-at-random mechanism or to ignore the incomplete records, and then to use the desired clustering method. We develop an efficient version of the $k$-means algorithm that allows for clustering cases where not all the features have observations recorded. Our extension is called $k_m$-means and reduces to the $k$-means algorithm when all records are complete. We also provide strategies to initialize our algorithm and to estimate the number of groups in the data set. Illustrations and simulations demonstrate the efficacy of our approach in a variety of settings and patterns of missing data. Our methods are also applied to the clustering of gamma-ray bursts and to the analysis of activation images obtained from a functional Magnetic Resonance Imaging experiment.

Read this paper on arXiv…

A. Lithio and R. Maitra
Mon, 26 Feb 18
36/49

Comments: 23 pages, 14 figures, 2 tables

Machine learning in APOGEE: Unsupervised spectral classification with $K$-means [IMA]

Posted on by

http://arxiv.org/abs/1801.07912


The data volume generated by astronomical surveys is growing rapidly. Traditional analysis techniques in spectroscopy either demand intensive human interaction or are computationally expensive. In this scenario, machine learning, and unsupervised clustering algorithms in particular offer interesting alternatives. The Apache Point Observatory Galactic Evolution Experiment (APOGEE) offers a vast data set of near-infrared stellar spectra which is perfect for testing such alternatives. Apply an unsupervised classification scheme based on $K$-means to the massive APOGEE data set. Explore whether the data are amenable to classification into discrete classes. We apply the $K$-means algorithm to 153,847 high resolution spectra ($R\approx22,500$). We discuss the main virtues and weaknesses of the algorithm, as well as our choice of parameters. We show that a classification based on normalised spectra captures the variations in stellar atmospheric parameters, chemical abundances, and rotational velocity, among other factors. The algorithm is able to separate the bulge and halo populations, and distinguish dwarfs, sub-giants, RC and RGB stars. However, a discrete classification in flux space does not result in a neat organisation in the parameters space. Furthermore, the lack of obvious groups in flux space causes the results to be fairly sensitive to the initialisation, and disrupts the efficiency of commonly-used methods to select the optimal number of clusters. Our classification is publicly available, including extensive online material associated with the APOGEE Data Release 12 (DR12). Our description of the APOGEE database can enormously help with the identification of specific types of targets for various applications. We find a lack of obvious groups in flux space, and identify limitations of the $K$-means algorithm in dealing with this kind of data.

Read this paper on arXiv…

R. Garcia-Dias, C. Prieto, J. Almeida, et. al.
Thu, 25 Jan 18
56/67

Comments: 23 pages, 24 images and online material

Improving Orbit Prediction Accuracy through Supervised Machine Learning [EPA]

Posted on by

http://arxiv.org/abs/1801.04856


Due to the lack of information such as the space environment condition and resident space objects’ (RSOs’) body characteristics, current orbit predictions that are solely grounded on physics-based models may fail to achieve required accuracy for collision avoidance and have led to satellite collisions already. This paper presents a methodology to predict RSOs’ trajectories with higher accuracy than that of the current methods. Inspired by the machine learning (ML) theory through which the models are learned based on large amounts of observed data and the prediction is conducted without explicitly modeling space objects and space environment, the proposed ML approach integrates physics-based orbit prediction algorithms with a learning-based process that focuses on reducing the prediction errors. Using a simulation-based space catalog environment as the test bed, the paper demonstrates three types of generalization capability for the proposed ML approach: 1) the ML model can be used to improve the same RSO’s orbit information that is not available during the learning process but shares the same time interval as the training data; 2) the ML model can be used to improve predictions of the same RSO at future epochs; and 3) the ML model based on a RSO can be applied to other RSOs that share some common features.

Read this paper on arXiv…

H. Peng and X. Bai
Tue, 16 Jan 18
35/79

Comments: 30 pages, 21 figures, 4 tables, Preprint submitted to Advances in Space Research, on December 14, 2017

Denoising Gravitational Waves using Deep Learning with Recurrent Denoising Autoencoders [CL]

Posted on by

http://arxiv.org/abs/1711.09919


Gravitational wave astronomy is a rapidly growing field of modern astrophysics, with observations being made frequently by the LIGO detectors. Gravitational wave signals are often extremely weak and the data from the detectors, such as LIGO, is contaminated with non-Gaussian and non-stationary noise, often containing transient disturbances which can obscure real signals. Traditional denoising methods, such as principal component analysis and dictionary learning, are not optimal for dealing with this non-Gaussian noise, especially for low signal-to-noise ratio gravitational wave signals. Furthermore, these methods are computationally expensive on large datasets. To overcome these issues, we apply state-of-the-art signal processing techniques, based on recent groundbreaking advancements in deep learning, to denoise gravitational wave signals embedded either in Gaussian noise or in real LIGO noise. We introduce SMTDAE, a Staired Multi-Timestep Denoising Autoencoder, based on sequence-to-sequence bi-directional Long-Short-Term-Memory recurrent neural networks. We demonstrate the advantages of using our unsupervised deep learning approach and show that, after training only using simulated Gaussian noise, SMTDAE achieves superior recovery performance for gravitational wave signals embedded in real non-Gaussian LIGO noise.

Read this paper on arXiv…

H. Shen, D. George, E. Huerta, et. al.
Wed, 29 Nov 17
17/69

Comments: 5 pages, 2 figures

Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation with Advanced LIGO Data [CL]

Posted on by

http://arxiv.org/abs/1711.07966


The recent Nobel-prize-winning detections of gravitational waves from merging black holes and the subsequent detection of the collision of two neutron stars in coincidence with electromagnetic observations have inaugurated a new era of multimessenger astrophysics. To enhance the scope of this emergent science, the use of deep convolutional neural networks were proposed for the detection and characterization of gravitational wave signals in real-time. This approach, Deep Filtering, was initially demonstrated using simulated LIGO noise. In this article, we present the extension of Deep Filtering using real noise from the first observing run of LIGO, for both detection and parameter estimation of gravitational waves from binary black hole mergers with continuous data streams from multiple LIGO detectors. We show for the first time that machine learning can detect and estimate the true parameters of a real GW event observed by LIGO. Our comparisons show that Deep Filtering is far more computationally efficient than matched-filtering, while retaining similar performance, allowing real-time processing of weak time-series signals in non-stationary non-Gaussian noise, with minimal resources, and also enables the detection of new classes of gravitational wave sources that may go unnoticed with existing detection algorithms. This framework is uniquely suited to enable coincident detection campaigns of gravitational waves and their multimessenger counterparts in real-time.

Read this paper on arXiv…

D. George and E. Huerta
Wed, 22 Nov 17
65/67

Comments: Version accepted to NIPS 2017 conference workshop on Deep Learning for Physical Sciences and selected for contributed talk. Also awarded 1st place at ACM SRC at SC17. This is a shorter version of arXiv:1711.03121

Glitch Classification and Clustering for LIGO with Deep Transfer Learning [IMA]

Posted on by

http://arxiv.org/abs/1711.07468


The detection of gravitational waves with LIGO and Virgo requires a detailed understanding of the response of these instruments in the presence of environmental and instrumental noise. Of particular interest is the study of non-Gaussian noise transients known as glitches, since their high occurrence rate in LIGO/Virgo data can obscure or even mimic true gravitational wave signals. Therefore, successfully identifying and excising glitches is of utmost importance to detect and characterize gravitational waves. In this article, we present the first application of Deep Learning combined with Transfer Learning for glitch classification, using real data from LIGO’s first discovery campaign labeled by Gravity Spy, showing that knowledge from pre-trained models for real-world object recognition can be transferred for classifying spectrograms of glitches. We demonstrate that this method enables the optimal use of very deep convolutional neural networks for glitch classification given small unbalanced training datasets, significantly reduces the training time, and achieves state-of-the-art accuracy above 98.8%. Once trained via transfer learning, we show that the networks can be truncated and used as feature extractors for unsupervised clustering to automatically group new classes of glitches. This feature is of critical importance to identify and remove new types of glitches which will occur as the LIGO/Virgo detectors gradually attain design sensitivity.

Read this paper on arXiv…

D. George, H. Shen and E. Huerta
Tue, 21 Nov 17
41/79

Comments: This is the short version accepted to NIPS 2017 conference workshop on Deep Learning for Physical Sciences. Extended article: arXiv:1706.07446

Deep Learning for Real-time Gravitational Wave Detection and Parameter Estimation: Results with Advanced LIGO Data [CL]

Posted on by

http://arxiv.org/abs/1711.03121


The recent Nobel-prize-winning detections of gravitational waves from merging black holes and the subsequent detection of the collision of two neutron stars in coincidence with electromagnetic observations have inaugurated a new era of multimessenger astrophysics. To enhance the scope of this emergent field of science, we pioneered the use of deep learning with convolutional neural networks, that take time-series inputs, for rapid detection and characterization of gravitational wave signals. This approach, Deep Filtering, was initially demonstrated using simulated LIGO noise. In this article, we present the extension of Deep Filtering using real data from LIGO, for both detection and parameter estimation of gravitational waves from binary black hole mergers using continuous data streams from multiple LIGO detectors. We demonstrate for the first time that machine learning can detect and estimate the true parameters of real events observed by LIGO. Our results show that Deep Filtering achieves similar sensitivities and lower errors compared to matched-filtering while being far more computationally efficient and more resilient to glitches, allowing real-time processing of weak time-series signals in non-stationary non-Gaussian noise with minimal resources, and also enables the detection of new classes of gravitational wave sources that may go unnoticed with existing detection algorithms. This unified framework for data analysis is ideally suited to enable coincident detection campaigns of gravitational waves and their multimessenger counterparts in real-time.

Read this paper on arXiv…

D. George and E. Huerta
Fri, 10 Nov 17
32/55

Comments: 6 pages, 7 figures; First application of deep learning to real LIGO events; Includes direct comparison against matched-filtering

Estimating Cosmological Parameters from the Dark Matter Distribution [CEA]

Posted on by

http://arxiv.org/abs/1711.02033


A grand challenge of the 21st century cosmology is to accurately estimate the cosmological parameters of our Universe. A major approach to estimating the cosmological parameters is to use the large-scale matter distribution of the Universe. Galaxy surveys provide the means to map out cosmic large-scale structure in three dimensions. Information about galaxy locations is typically summarized in a “single” function of scale, such as the galaxy correlation function or power-spectrum. We show that it is possible to estimate these cosmological parameters directly from the distribution of matter. This paper presents the application of deep 3D convolutional networks to volumetric representation of dark-matter simulations as well as the results obtained using a recently proposed distribution regression framework, showing that machine learning techniques are comparable to, and can sometimes outperform, maximum-likelihood point estimates using “cosmological models”. This opens the way to estimating the parameters of our Universe with higher accuracy.

Read this paper on arXiv…

S. Ravanbakhsh, J. Oliva, S. Fromenteau, et. al.
Tue, 7 Nov 17
62/118

Comments: ICML 2016

The Fog of War: A Machine Learning Approach to Forecasting Weather on Mars [IMA]

Posted on by

http://arxiv.org/abs/1706.08915


For over a decade, scientists at NASA’s Jet Propulsion Laboratory (JPL) have been recording measurements from the Martian surface as a part of the Mars Exploration Rovers mission. One quantity of interest has been the opacity of Mars’s atmosphere for its importance in day-to-day estimations of the amount of power available to the rover from its solar arrays. This paper proposes the use of neural networks as a method for forecasting Martian atmospheric opacity that is more effective than the current empirical model. The more accurate prediction provided by these networks would allow operators at JPL to make more accurate predictions of the amount of energy available to the rover when they plan activities for coming sols.

Read this paper on arXiv…

D. Bellutta
Wed, 28 Jun 17
-31/62

Comments: N/A

Deep Transfer Learning: A new deep learning glitch classification method for advanced LIGO [CL]

Posted on by

http://arxiv.org/abs/1706.07446


The exquisite sensitivity of the advanced LIGO detectors has enabled the detection of multiple gravitational wave signals. The sophisticated design of these detectors mitigates the effect of most types of noise. However, advanced LIGO data streams are contaminated by numerous artifacts known as glitches: non-Gaussian noise transients with complex morphologies. Given their high rate of occurrence, glitches can lead to false coincident detections, obscure and even mimic gravitational wave signals. Therefore, successfully characterizing and removing glitches from advanced LIGO data is of utmost importance. Here, we present the first application of Deep Transfer Learning for glitch classification, showing that knowledge from deep learning algorithms trained for real-world object recognition can be transferred for classifying glitches in time-series based on their spectrogram images. Using the Gravity Spy dataset, containing hand-labeled, multi-duration spectrograms obtained from real LIGO data, we demonstrate that this method enables optimal use of very deep convolutional neural networks for classification given small training datasets, significantly reduces the time for training the networks, and achieves state-of-the-art accuracy above 98.8%, with perfect precision-recall on 8 out of 22 classes. Furthermore, new types of glitches can be classified accurately given few labeled examples with this technique. Once trained via transfer learning, we show that the convolutional neural networks can be truncated and used as excellent feature extractors for unsupervised clustering methods to identify new classes based on their morphology, without any labeled examples. Therefore, this provides a new framework for dynamic glitch classification for gravitational wave detectors, which are expected to encounter new types of noise as they undergo gradual improvements to attain design sensitivity.

Read this paper on arXiv…

D. George, H. Shen and E. Huerta
Mon, 26 Jun 17
9/40

Comments: N/A

A hybrid supervised/unsupervised machine learning approach to solar flare prediction [SSA]

Posted on by

http://arxiv.org/abs/1706.07103


We introduce a hybrid approach to solar flare prediction, whereby a supervised regularization method is used to realize feature importance and an unsupervised clustering method is used to realize the binary flare/no-flare decision. The approach is validated against NOAA SWPC data.

Read this paper on arXiv…

F. Benvenuto, M. Piana, C. Campi, et. al.
Fri, 23 Jun 17
42/48

Comments: N/A

Creating Virtual Universes Using Generative Adversarial Networks [IMA]

Posted on by

http://arxiv.org/abs/1706.02390


Inferring model parameters from experimental data is a grand challenge in many sciences, including cosmology. This often relies critically on high fidelity numerical simulations, which are prohibitively computationally expensive. The application of deep learning techniques to generative modeling is renewing interest in using high dimensional density estimators as computationally inexpensive emulators of fully-fledged simulations. These generative models have the potential to make a dramatic shift in the field of scientific simulations, but for that shift to happen we need to study the performance of such generators in the precision regime needed for science applications. To this end, in this letter we apply Generative Adversarial Networks to the problem of generating cosmological weak lensing convergence maps. We show that our generator network produces maps that are described by, with high statistical confidence, the same summary statistics as the fully simulated maps.

Read this paper on arXiv…

M. Mustafa, D. Bard, W. Bhimji, et. al.
Fri, 9 Jun 17
46/52

Comments: 8 pages, 5 figures

Maximum Likelihood Estimation based on Random Subspace EDA: Application to Extrasolar Planet Detection [CL]

Posted on by

http://arxiv.org/abs/1704.05761


This paper addresses maximum likelihood (ML) estimation based model fitting in the context of extrasolar planet detection. This problem is featured by the following properties: 1) the candidate models under consideration are highly nonlinear; 2) the likelihood surface has a huge number of peaks; 3) the parameter space ranges in size from a few to dozens of dimensions. These properties make the ML search a very challenging problem, as it lacks any analytical or gradient based searching solution to explore the parameter space. A population based searching method, called estimation of distribution algorithm (EDA) is adopted to explore the model parameter space starting from a batch of random locations. EDA is featured by its ability to reveal and utilize problem structures. This property is desirable for characterizing the detections. However, it is well recognized that EDAs can not scale well to large scale problems, as it consists of iterative random sampling and model fitting procedures, which results in the well-known dilemma curse of dimensionality. A novel mechanism to perform EDAs in interactive random subspaces spanned by correlated variables is proposed. This mechanism is totally adaptive and is capable of alleviating curse of dimensionality for EDAs to a large extent, as the dimension of each subspace is much smaller than that of the full parameter space. The efficiency of the proposed algorithm is verified via both benchmark numerical studies and real data analysis.

Read this paper on arXiv…

B. Liu and K. Chen
Thu, 20 Apr 17
28/49

Comments: 12 pages, 5 figures, conference

Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit [IMA]

Posted on by

http://arxiv.org/abs/1702.00403


Observations of astrophysical objects such as galaxies are limited by various sources of random and systematic noise from the sky background, the optical system of the telescope and the detector used to record the data. Conventional deconvolution techniques are limited in their ability to recover features in imaging data by the Shannon-Nyquist sampling theorem. Here we train a generative adversarial network (GAN) on a sample of $4,550$ images of nearby galaxies at $0.01<z<0.02$ from the Sloan Digital Sky Survey and conduct $10\times$ cross validation to evaluate the results. We present a method using a GAN trained on galaxy images that can recover features from artificially degraded images with worse seeing and higher noise than the original with a performance which far exceeds simple deconvolution. The ability to better recover detailed features such as galaxy morphology from low-signal-to-noise and low angular resolution imaging data significantly increases our ability to study existing data sets of astrophysical objects as well as future observations with observatories such as the Large Synoptic Sky Telescope (LSST) and the Hubble and James Webb space telescopes.

Read this paper on arXiv…

K. Schawinski, C. Zhang, H. Zhang, et. al.
Fri, 3 Feb 17
34/55

Comments: Accepted for publication in MNRAS, for the full code and a virtual machine set up to run it, see this http URL

FPGA Architecture for Deep Learning and its application to Planetary Robotics [CL]

Posted on by

http://arxiv.org/abs/1701.07543


Autonomous control systems onboard planetary rovers and spacecraft benefit from having cognitive capabilities like learning so that they can adapt to unexpected situations in-situ. Q-learning is a form of reinforcement learning and it has been efficient in solving certain class of learning problems. However, embedded systems onboard planetary rovers and spacecraft rarely implement learning algorithms due to the constraints faced in the field, like processing power, chip size, convergence rate and costs due to the need for radiation hardening. These challenges present a compelling need for a portable, low-power, area efficient hardware accelerator to make learning algorithms practical onboard space hardware. This paper presents a FPGA implementation of Q-learning with Artificial Neural Networks (ANN). This method matches the massive parallelism inherent in neural network software with the fine-grain parallelism of an FPGA hardware thereby dramatically reducing processing time. Mars Science Laboratory currently uses Xilinx-Space-grade Virtex FPGA devices for image processing, pyrotechnic operation control and obstacle avoidance. We simulate and program our architecture on a Xilinx Virtex 7 FPGA. The architectural implementation for a single neuron Q-learning and a more complex Multilayer Perception (MLP) Q-learning accelerator has been demonstrated. The results show up to a 43-fold speed up by Virtex 7 FPGAs compared to a conventional Intel i5 2.3 GHz CPU. Finally, we simulate the proposed architecture using the Symphony simulator and compiler from Xilinx, and evaluate the performance and power consumption.

Read this paper on arXiv…

P. Gankidi and J. Thangavelautham
Fri, 27 Jan 17
35/54

Comments: 8 pages, 10 figures in Proceedings of the IEEE Aerospace Conference 2017

Deep Neural Networks to Enable Real-time Multimessenger Astrophysics [IMA]

Posted on by

http://arxiv.org/abs/1701.00008


We introduce a new methodology for time-domain signal processing, based on deep learning neural networks, which has the potential to revolutionize data analysis in science. To illustrate how this enables real-time multimessenger astrophysics, we designed two deep convolutional neural networks that can analyze time-series data from observatories including advanced LIGO. The first neural network recognizes the presence of gravitational waves from binary black hole mergers, and the second one estimates the mass of each black hole, given weak signals hidden in extremely noisy time-series inputs. We highlight the advantages offered by this novel method, which outperforms matched-filtering or conventional machine learning techniques, and propose strategies to extend our implementation for simultaneously targeting different classes of gravitational wave sources while ignoring anomalous noise transients. Our results strongly indicate that deep neural networks are highly efficient and versatile tools for directly processing raw noisy data streams. Furthermore, we pioneer a new paradigm to accelerate scientific discovery by combining high-performance simulations on traditional supercomputers and artificial intelligence algorithms that exploit innovative hardware architectures such as deep-learning-optimized GPUs. This unique approach immediately provides a natural framework to unify multi-spectrum observations in real-time, thus enabling coincident detection campaigns of gravitational waves sources and their electromagnetic counterparts.

Read this paper on arXiv…

D. George and E. Huerta
Tue, 3 Jan 17
34/55

Comments: 20 pages, 15 figures

Correlated signal inference by free energy exploration [CL]

Posted on by

http://arxiv.org/abs/1612.08406


The inference of correlated signal fields with unknown correlation structures is of high scientific and technological relevance, but poses significant conceptual and numerical challenges. To address these, we develop the correlated signal inference (CSI) algorithm within information field theory (IFT) and discuss its numerical implementation. To this end, we introduce the free energy exploration (FrEE) strategy for numerical information field theory (NIFTy) applications. The FrEE strategy is to let the mathematical structure of the inference problem determine the dynamics of the numerical solver. FrEE uses the Gibbs free energy formalism for all involved unknown fields and correlation structures without marginalization of nuisance quantities. It thereby avoids the complexity marginalization often impose to IFT equations. FrEE simultaneously solves for the mean and the uncertainties of signal, nuisance, and auxiliary fields, while exploiting any analytically calculable quantity. Finally, FrEE uses a problem specific and self-tuning exploration strategy to swiftly identify the optimal field estimates as well as their uncertainty maps. For all estimated fields, properly weighted posterior samples drawn from their exact, fully non-Gaussian distributions can be generated. Here, we develop the FrEE strategies for the CSI of a normal, a log-normal, and a Poisson log-normal IFT signal inference problem and demonstrate their performances via their NIFTy implementations.

Read this paper on arXiv…

T. Ensslin and J. Knollmuller
Wed, 28 Dec 16
31/46

Comments: 19 pages, 5 figures, submitted

Learning an Astronomical Catalog of the Visible Universe through Scalable Bayesian Inference [CL]

Posted on by

http://arxiv.org/abs/1611.03404


Celeste is a procedure for inferring astronomical catalogs that attains state-of-the-art scientific results. To date, Celeste has been scaled to at most hundreds of megabytes of astronomical images: Bayesian posterior inference is notoriously demanding computationally. In this paper, we report on a scalable, parallel version of Celeste, suitable for learning catalogs from modern large-scale astronomical datasets. Our algorithmic innovations include a fast numerical optimization routine for Bayesian posterior inference and a statistically efficient scheme for decomposing astronomical optimization problems into subproblems.
Our scalable implementation is written entirely in Julia, a new high-level dynamic programming language designed for scientific and numerical computing. We use Julia’s high-level constructs for shared and distributed memory parallelism, and demonstrate effective load balancing and efficient scaling on up to 8192 Xeon cores on the NERSC Cori supercomputer.

Read this paper on arXiv…

J. Regier, K. Pamnany, R. Giordano, et. al.
Fri, 11 Nov 16
11/40

Comments: submitting to IPDPS’17

Deep Recurrent Neural Networks for Supernovae Classification [IMA]

Posted on by

http://arxiv.org/abs/1606.07442


We apply deep recurrent neural networks, which are capable of learning complex sequential information, to classify supernovae. The observational time and filter fluxes are used as inputs to the network, but since the inputs are agnostic additional data such as host galaxy information can also be included. Using the Supernovae Photometric Classification Challenge (SPCC) data, we find that deep networks are capable of learning about light curves, however the performance of the network is highly sensitive to the amount of training data. For a training size of 50% of the representational SPCC dataset (around 104 supernovae) we obtain a type Ia vs non type Ia classification accuracy of 94.8%, an area under the Receiver Operating Characteristic curve AUC of 0.986 and a SPCC figure-of-merit F1 = 0.64. We also apply a pre-trained model to obtain classification probabilities as a function of time, and show it can give early indications of supernovae type. Our method is competitive with existing algorithms and has applications for future large-scale photometric surveys.

Read this paper on arXiv…

T. Charnock and A. Moss
Mon, 27 Jun 16
27/43

Comments: 6 pages, 3 figures

A Deep-Learning Approach for Operation of an Automated Realtime Flare Forecast [SSA]

Posted on by

http://arxiv.org/abs/1606.01587


Automated forecasts serve important role in space weather science, by providing statistical insights to flare-trigger mechanisms, and by enabling tailor-made forecasts and high-frequency forecasts. Only by realtime forecast we can experimentally measure the performance of flare-forecasting methods while confidently avoiding overlearning.
We have been operating unmanned flare forecast service since August, 2015 that provides 24-hour-ahead forecast of solar flares, every 12 minutes. We report the method and prediction results of the system.

Read this paper on arXiv…

Y. Hada-Muranushi, T. Muranushi, A. Asai, et. al.
Tue, 7 Jun 16
13/80

Comments: 6 pages, 4 figures

Practical Introduction to Clustering Data [CL]

Posted on by

http://arxiv.org/abs/1602.05124


Data clustering is an approach to seek for structure in sets of complex data, i.e., sets of “objects”. The main objective is to identify groups of objects which are similar to each other, e.g., for classification. Here, an introduction to clustering is given and three basic approaches are introduced: the k-means algorithm, neighbour-based clustering, and an agglomerative clustering method. For all cases, C source code examples are given, allowing for an easy implementation.

Read this paper on arXiv…

A. Hartmann
Wed, 17 Feb 16
30/55

Comments: 22 pages. All source code in anc directory included. Section 8.5.6 of book: A.K. Hartmann, Big Practical Guide to Computer Simulations, World-Scientifc, Singapore (2015)

Generation of a Supervised Classification Algorithm for Time-Series Variable Stars with an Application to the LINEAR Dataset [IMA]

Posted on by

http://arxiv.org/abs/1601.03769


With the advent of digital astronomy, new benefits and new problems have been presented to the modern day astronomer. While data can be captured in a more efficient and accurate manor using digital means, the efficiency of data retrieval has led to an overload of scientific data for processing and storage. This paper will focus on the construction and application of a supervised pattern classification algorithm for the identification of variable stars. Given the reduction of a survey of stars into a standard feature space, the problem of using prior patterns to identify new observed patterns can be reduced to time tested classification methodologies and algorithms. Such supervised methods, so called because the user trains the algorithms prior to application using patterns with known classes or labels, provide a means to probabilistically determine the estimated class type of new observations. This paper will demonstrate the construction and application of a supervised classification algorithm on variable star data. The classifier is applied to a set of 192,744 LINEAR data points. Of the original samples, 34,451 unique stars were classified with high confidence (high level of probability of being the true class).

Read this paper on arXiv…

K. Johnston and H. Oluseyi
Mon, 18 Jan 16
25/50

Comments: N/A

Computational Intelligence Challenges and Applications on Large-Scale Astronomical Time Series Databases [IMA]

Posted on by

http://arxiv.org/abs/1509.07823


Time-domain astronomy (TDA) is facing a paradigm shift caused by the exponential growth of the sample size, data complexity and data generation rates of new astronomical sky surveys. For example, the Large Synoptic Survey Telescope (LSST), which will begin operations in northern Chile in 2022, will generate a nearly 150 Petabyte imaging dataset of the southern hemisphere sky. The LSST will stream data at rates of 2 Terabytes per hour, effectively capturing an unprecedented movie of the sky. The LSST is expected not only to improve our understanding of time-varying astrophysical objects, but also to reveal a plethora of yet unknown faint and fast-varying phenomena. To cope with a change of paradigm to data-driven astronomy, the fields of astroinformatics and astrostatistics have been created recently. The new data-oriented paradigms for astronomy combine statistics, data mining, knowledge discovery, machine learning and computational intelligence, in order to provide the automated and robust methods needed for the rapid detection and classification of known astrophysical objects as well as the unsupervised characterization of novel phenomena. In this article we present an overview of machine learning and computational intelligence applications to TDA. Future big data challenges and new lines of research in TDA, focusing on the LSST, are identified and discussed from the viewpoint of computational intelligence/machine learning. Interdisciplinary collaboration will be required to cope with the challenges posed by the deluge of astronomical data coming from the LSST.

Read this paper on arXiv…

P. Huijse, P. Estevez, P. Protopapas, et. al.
Mon, 28 Sep 15
26/67

Comments: N/A

A review of learning vector quantization classifiers [CL]

Posted on by

http://arxiv.org/abs/1509.07093


In this work we present a review of the state of the art of Learning Vector Quantization (LVQ) classifiers. A taxonomy is proposed which integrates the most relevant LVQ approaches to date. The main concepts associated with modern LVQ approaches are defined. A comparison is made among eleven LVQ classifiers using one real-world and two artificial datasets.

Read this paper on arXiv…

D. Nova and P. Estevez
Thu, 24 Sep 15
53/60

Comments: 14 pages

Sensitivity study using machine learning algorithms on simulated r-mode gravitational wave signals from newborn neutron stars [IMA]

Posted on by

http://arxiv.org/abs/1508.02064


This is a follow-up sensitivity study on r-mode gravitational wave signals from newborn neutron stars illustrating the applicability of machine learning algorithms for the detection of long-lived gravitational-wave transients. In this sensitivity study we examine three machine learning algorithms (MLAs): artificial neural networks (ANNs), support vector machines (SVMs) and constrained subspace classifiers (CSCs). The objective of this study is to compare the detection efficiency that MLAs can achieve with the efficiency of conventional detection algorithms discussed in an earlier paper. Comparisons are made using 2 distinct r-mode waveforms. For the training of the MLAs we assumed that some information about the distance to the source is given so that the training was performed over distance ranges not wider than half an order of magnitude. The results of this study suggest that machine learning algorithms are suitable for the detection of long-lived gravitational-wave transients and that when assuming knowledge of the distance to the source, MLAs are at least as efficient as conventional methods.

Read this paper on arXiv…

A. Mytidis, A. Panagopoulos, O. Panagopoulos, et. al.
Tue, 11 Aug 15
12/57

Comments: N/A

Removing systematic errors for exoplanet search via latent causes [CL]

Posted on by

http://arxiv.org/abs/1505.03036


We describe a method for removing the effect of confounders in order to reconstruct a latent quantity of interest. The method, referred to as half-sibling regression, is inspired by recent work in causal inference using additive noise models. We provide a theoretical justification and illustrate the potential of the method in a challenging astronomy application.

Read this paper on arXiv…

B. Scholkopf, D. Hogg, D. Wang, et. al.
Wed, 13 May 15
6/69

Comments: Extended version of a paper appearing in the Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 2015

ASTROMLSKIT: A New Statistical Machine Learning Toolkit: A Platform for Data Analytics in Astronomy [CL]

Posted on by

http://arxiv.org/abs/1504.07865


Astroinformatics is a new impact area in the world of astronomy, occasionally called the final frontier, where several astrophysicists, statisticians and computer scientists work together to tackle various data intensive astronomical problems. Exponential growth in the data volume and increased complexity of the data augments difficult questions to the existing challenges. Classical problems in Astronomy are compounded by accumulation of astronomical volume of complex data, rendering the task of classification and interpretation incredibly laborious. The presence of noise in the data makes analysis and interpretation even more arduous. Machine learning algorithms and data analytic techniques provide the right platform for the challenges posed by these problems. A diverse range of open problem like star-galaxy separation, detection and classification of exoplanets, classification of supernovae is discussed. The focus of the paper is the applicability and efficacy of various machine learning algorithms like K Nearest Neighbor (KNN), random forest (RF), decision tree (DT), Support Vector Machine (SVM), Na\”ive Bayes and Linear Discriminant Analysis (LDA) in analysis and inference of the decision theoretic problems in Astronomy. The machine learning algorithms, integrated into ASTROMLSKIT, a toolkit developed in the course of the work, have been used to analyze HabCat data and supernovae data. Accuracy has been found to be appreciably good.

Read this paper on arXiv…

S. Saha, S. Agrawal, M. R, et. al.
Thu, 30 Apr 15
43/43

Comments: Habitability Catalog (HabCat), Supernova classification, data analysis, Astroinformatics, Machine learning, ASTROMLS toolkit, Na\”ive Bayes, SVD, PCA, Random Forest, SVM, Decision Tree, LDA

Meta learning of bounds on the Bayes classifier error [CL]

Posted on by

http://arxiv.org/abs/1504.07116


Meta learning uses information from base learners (e.g. classifiers or estimators) as well as information about the learning problem to improve upon the performance of a single base learner. For example, the Bayes error rate of a given feature space, if known, can be used to aid in choosing a classifier, as well as in feature selection and model selection for the base classifiers and the meta classifier. Recent work in the field of f-divergence functional estimation has led to the development of simple and rapidly converging estimators that can be used to estimate various bounds on the Bayes error. We estimate multiple bounds on the Bayes error using an estimator that applies meta learning to slowly converging plug-in estimators to obtain the parametric convergence rate. We compare the estimated bounds empirically on simulated data and then estimate the tighter bounds on features extracted from an image patch analysis of sunspot continuum and magnetogram images.

Read this paper on arXiv…

K. Moon, V. Delouille and A. Hero
Tue, 28 Apr 15
36/70

Comments: 6 pages, 3 figures

Rotation-invariant convolutional neural networks for galaxy morphology prediction [IMA]

Posted on by

http://arxiv.org/abs/1503.07077


Measuring the morphological parameters of galaxies is a key requirement for studying their formation and evolution. Surveys such as the Sloan Digital Sky Survey (SDSS) have resulted in the availability of very large collections of images, which have permitted population-wide analyses of galaxy morphology. Morphological analysis has traditionally been carried out mostly via visual inspection by trained experts, which is time-consuming and does not scale to large ($\gtrsim10^4$) numbers of images.
Although attempts have been made to build automated classification systems, these have not been able to achieve the desired level of accuracy. The Galaxy Zoo project successfully applied a crowdsourcing strategy, inviting online users to classify images by answering a series of questions. Unfortunately, even this approach does not scale well enough to keep up with the increasing availability of galaxy images.
We present a deep neural network model for galaxy morphology classification which exploits translational and rotational symmetry. It was developed in the context of the Galaxy Challenge, an international competition to build the best model for morphology classification based on annotated images from the Galaxy Zoo project.
For images with high agreement among the Galaxy Zoo participants, our model is able to reproduce their consensus with near-perfect accuracy ($> 99\%$) for most questions. Confident model predictions are highly accurate, which makes the model suitable for filtering large collections of images and forwarding challenging images to experts for manual annotation. This approach greatly reduces the experts’ workload without affecting accuracy. The application of these algorithms to larger sets of training data will be critical for analysing results from future surveys such as the LSST.

Read this paper on arXiv…

S. Dieleman, K. Willett and J. Dambre
Wed, 25 Mar 15
30/38

Comments: Accepted for publication in MNRAS. 20 pages, 14 figures

Combining human and machine learning for morphological analysis of galaxy images [IMA]

Posted on by

http://arxiv.org/abs/1409.7935

The increasing importance of digital sky surveys collecting many millions of galaxy images has reinforced the need for robust methods that can perform morphological analysis of large galaxy image databases. Citizen science initiatives such as Galaxy Zoo showed that large datasets of galaxy images can be analyzed effectively by non-scientist volunteers, but since databases generated by robotic telescopes grow much faster than the processing power of any group of citizen scientists, it is clear that computer analysis is required. Here we propose to use citizen science data for training machine learning systems, and show experimental results demonstrating that machine learning systems can be trained with citizen science data. Our findings show that the performance of machine learning depends on the quality of the data, which can be improved by using samples that have a high degree of agreement between the citizen scientists. The source code of the method is publicly available.

Read this paper on arXiv…

E. Kuminski, J. George, J. Wallin, et. al.
Tue, 30 Sep 14
6/81

Comments: PASP, accepted

Hellinger Distance Trees for Imbalanced Streams [CL]

Posted on by

http://arxiv.org/abs/1405.2278

Classifiers trained on data sets possessing an imbalanced class distribution are known to exhibit poor generalisation performance. This is known as the imbalanced learning problem. The problem becomes particularly acute when we consider incremental classifiers operating on imbalanced data streams, especially when the learning objective is rare class identification. As accuracy may provide a misleading impression of performance on imbalanced data, existing stream classifiers based on accuracy can suffer poor minority class performance on imbalanced streams, with the result being low minority class recall rates. In this paper we address this deficiency by proposing the use of the Hellinger distance measure, as a very fast decision tree split criterion. We demonstrate that by using Hellinger a statistically significant improvement in recall rates on imbalanced data streams can be achieved, with an acceptable increase in the false positive rate.

Read this paper on arXiv…

R. Lyon, J. Brooke, J. Knowles, et. al.
Mon, 12 May 14
22/40

Comments: 6 Pages, 2 figures, to be published in Proceedings 22nd International Conference on Pattern Recognition (ICPR) 2014

Supervised detection of anomalous light-curves in massive astronomical catalogs [CL]

Posted on by

http://arxiv.org/abs/1404.4888

The development of synoptic sky surveys has led to a massive amount of data for which resources needed for analysis are beyond human capabilities. To process this information and to extract all possible knowledge, machine learning techniques become necessary. Here we present a new method to automatically discover unknown variable objects in large astronomical catalogs. With the aim of taking full advantage of all the information we have about known objects, our method is based on a supervised algorithm. In particular, we train a random forest classifier using known variability classes of objects and obtain votes for each of the objects in the training set. We then model this voting distribution with a Bayesian network and obtain the joint voting distribution among the training objects. Consequently, an unknown object is considered as an outlier insofar it has a low joint probability. Our method is suitable for exploring massive datasets given that the training process is performed offline. We tested our algorithm on 20 millions light-curves from the MACHO catalog and generated a list of anomalous candidates. We divided the candidates into two main classes of outliers: artifacts and intrinsic outliers. Artifacts were principally due to air mass variation, seasonal variation, bad calibration or instrumental errors and were consequently removed from our outlier list and added to the training set. After retraining, we selected about 4000 objects, which we passed to a post analysis stage by perfoming a cross-match with all publicly available catalogs. Within these candidates we identified certain known but rare objects such as eclipsing Cepheids, blue variables, cataclysmic variables and X-ray sources. For some outliers there were no additional information. Among them we identified three unknown variability types and few individual outliers that will be followed up for a deeper analysis.

Read this paper on arXiv…

I. nun, K. Pichara, P. Protopapas, et. al.
Tue, 22 Apr 14
31/54

SOMz: photometric redshift PDFs with self organizing maps and random atlas [IMA]

Posted on by

http://arxiv.org/abs/1312.5753

In this paper we explore the applicability of the unsupervised machine learning technique of Self Organizing Maps (SOM) to estimate galaxy photometric redshift probability density functions (PDFs). This technique takes a spectroscopic training set, and maps the photometric attributes, but not the redshifts, to a two dimensional surface by using a process of competitive learning where neurons compete to more closely resemble the training data multidimensional space. The key feature of a SOM is that it retains the topology of the input set, revealing correlations between the attributes that are not easily identified. We test three different 2D topological mapping: rectangular, hexagonal, and spherical, by using data from the DEEP2 survey. We also explore different implementations and boundary conditions on the map and also introduce the idea of a random atlas where a large number of different maps are created and their individual predictions are aggregated to produce a more robust photometric redshift PDF. We also introduced a new metric, the $I$-score, which efficiently incorporates different metrics, making it easier to compare different results (from different parameters or different photometric redshift codes). We find that by using a spherical topology mapping we obtain a better representation of the underlying multidimensional topology, which provides more accurate results that are comparable to other, state-of-the-art machine learning algorithms. Our results illustrate that unsupervised approaches have great potential for many astronomical problems, and in particular for the computation of photometric redshifts.

Read this paper on arXiv…

Mon, 23 Dec 13
31/48

Automatic Classification of Variable Stars in Catalogs with missing data [IMA]

Posted on by

http://arxiv.org/abs/1310.7868

We present an automatic classification method for astronomical catalogs with missing data. We use Bayesian networks, a probabilistic graphical model, that allows us to perform inference to pre- dict missing values given observed data and dependency relationships between variables. To learn a Bayesian network from incomplete data, we use an iterative algorithm that utilises sampling methods and expectation maximization to estimate the distributions and probabilistic dependencies of variables from data with missing values. To test our model we use three catalogs with missing data (SAGE, 2MASS and UBVI) and one complete catalog (MACHO). We examine how classification accuracy changes when information from missing data catalogs is included, how our method compares to traditional missing data approaches and at what computational cost. Integrating these catalogs with missing data we find that classification of variable objects improves by few percent and by 15% for quasar detection while keeping the computational cost the same.

Read this paper on arXiv…

Date added: Wed, 30 Oct 13