Deep Learning @ INAF

16 Sept 2019, 13:00 → 19 Sept 2019, 17:00 Europe/Rome

Riccardo Smareglia (chair) (Istituto Nazionale di Astrofisica (INAF)), Massimo Brescia (Istituto Nazionale di Astrofisica (INAF)), Andrea Bulgarelli (Istituto Nazionale di Astrofisica (INAF)), Vito Conforti (Istituto Nazionale di Astrofisica (INAF)), Marco Landoni, Massimo Meneghetti (Istituto Nazionale di Astrofisica (INAF))

Astronomy is at the forefront of Big Data science, with exponentially growing data volumes and rates, and an ever-increasing complexity, now entering the Peta scale regime. This workshop has two main goals: to provide a theoretical and pragmatic view of the state-of-the-art and, second, to focus on the analysis and discussions about the problems of balancing the incoming data deluge with well-proportioned data mining and machine (deep) learning solutions.

 

Topics covered:

• Data Preparation and Exploration
• Basic theory and definitions
• Machine Learning paradigms (supervised and unsupervised learning)
• Regression problems
• Classification problems
• Deep Learning
• Machine Learning as a Service (MLaaS)

You will also get:

• Valuable Hands-on sessions are foreseen!
• Invited talks focused on applications of Deep/Machine Learning in Astrophysics (please check here for the prerequisites before arriving in Pula)

General overview of the meeting during its days:

Monday will be basically devoted as a buffer in order to allow people to travel on that day and then start early on Tuesday at 9 am sharp. There will an be open discussions about data science and machine learning problems that INAF researchers are facing. Networking between partecipants is encouraged.

• Tuesday Morning Prof. Claudio Gentile (Google Inc. Researcher in New York) will give lectures in which he will illustrate basic machine learning concepts starting from main definitions  (e.g. Training Set) up to main algorithms for pattern classifications both supervised and unsupervised.
• Tuesday Afternoon will be dedicated to Neural Network and introduction to Deep Learning with hands-on tutorial (Python).
• On Wednesday we will move on Data Exploration and preparation both from  the practical point of view (Python tutorial) and theoretical concepts (such as PCA concepts, Autoencoders, dimensionality reduction and Data Exploration) 
• Thursday will be devoted to tutorials and introduction to commercial platform for Machine Learning. In particular we will have lessons given by official teachers from Amazon (AWS SageMaker), IBM and Mathworks (Matlab toolbox for Machine Learning).

20190918_111318.jpg

form deep learning 2019-1.pdf

keras_tensorflow_instructions.pdf

Monday 16 September

Registration

BrainStorming

15:50 Coffee Break

BrainStorming

Tuesday 17 September

1 Welcome and info

Course 1a : Claudio Gentile ( Google researcher)

The regression in machine learning
The classification in machine learning
Types of classification algorithms in Machine Learning
(Naive Bayes Classifier LM, K-Nearest Neighbor, Decision Trees, Random Forest).

C.Gentile.pdf

10:50 Coffee Break

Course 1b : Claudio Gentile ( Google researcher)

Unsupervised learning (KMeans, SOM, KDTREE)
Linear classifier - Perceptron.
Introduction to the Aritificial neural network (MLP, Backpropagation)

13:00 Lunch

Course 2a: Introduction to Deep Learning

Convener: Dr Giuseppe Angora

16:00 Coffee Break

Course 2b: Introduction to Deep Learning - tutorial & practice

Convener: Dr Giuseppe Angora

Wednesday 18 September

Course 3a: Data pre & post processing - Parameter Space exploration

Conveners: Stefano Cavuoti (Università degli studi di Napoli, Federico II), Massimo Brescia (Istituto Nazionale di Astrofisica (INAF))

11:00 Coffee Break

Course 3b:: Data exploration - Tutorial & Practice

Conveners: Stefano Cavuoti (Università degli studi di Napoli, Federico II), Massimo Brescia (Istituto Nazionale di Astrofisica (INAF))

13:00 Lunch

Session 1: Data Science and Astrophysics - use case examples

Convener: Chair: Vito Conforti (Istituto Nazionale di Astrofisica (INAF))

2 (A few) Deep-Learning applications in Gravitational Lensing

Large scale imaging surveys will increase the number of galaxy-scale strong lensing candidates by maybe three orders of magnitudes beyond the number known today. Finding these rare objects will require picking them out of at least tens of millions of images and deriving scientific results from them will require quantifying the efficiency and bias of any search method. I present a description and results of an open gravitational lens finding challenge. Participants were
asked to identify lenses in simulated lenses. A variety of methods were used including visual inspection, arc and ring finders, support vector machines (SVM) and convolutional neural networks (CNN).
I will also discuss the future challenges and developments in this subject.

Speaker: Massimo Meneghetti (Istituto Nazionale di Astrofisica (INAF))

3 Machine Learning as a Service - Application of Google Cloud Platform to Machine Learning problems

The availability of data to astronomers is increasing exponentially every day and a strong synergy between machine learning (ML) algorithms, Big Data and proper computational environment is mandatory. In this context, Cloud Platforms could make a difference allowing to exploit the proper tools of the Data Science while offering the right computational environment for Machine Learning inferences in a BigData fashion just with few clicks. I will review how the Google Cloud Platform could enable astronomers to take the advantages of machine learning (in BigData regime) in an "as-a-service" fashion (MLaaS). I will illustrate the offered tools starting from the data preparation and Data Cleaning (Cloud Dataprep) and off-the shelf machine learning algorithms both for classification and regression problems. The focus of this contribution will be oriented in order to let the people aware of what kind MlaaS services are available and how to exploit them for real astronomer's life problems.

Speaker: Marco Landoni (Istituto Nazionale di Astrofisica (INAF))

PulaTalkLandoni2019.pptx

4 A Deep Learning approach for AGILE-GRID GRB Detection

This work presents a Deep Neural Network (DNN) approach for the detection of GRBs notified by external instruments in the AGILE-GRID energy range, between 100 MeV and 10 GeV, where time and position of the GRB is known in advance. Taking into account the complex observation pattern of AGILE, we developed a Convolutional Neural Network (CNN), a class of DNN mainly used for image classification, and we trained it with datasets of simulated AGILE-GRID maps. Half of these maps were simulated with background-only, and the other half with background and a GRB. The GRB model has been derived from the first Fermi-GBM catalogue, convolved with the AGILE-GRID exposure variation. After the CNN training phase, we tested the flexibility of this trained CNN with different observing models and conditions (based on the integrated exposure value and value of the background). For each condition, we provide a p-value distribution, used to define the thresholds of the CNN classification values for the different statistical significance level. The CNN has been applied to Fermi-GBM, Fermi-LAT and Swift-BAT catalogues: more than 20 AGILE-GRID counterparts above three sigma level are found. We also compared results obtained with the Li&Ma traditional method; the trained CNN is more flexible because the analysis is not constrained by the background time window selection.

Speaker: Nicolo' Parmiggiani (Istituto Nazionale di Astrofisica (INAF))

Parmiggiani-A Deep Learning approach for AGILE-GRID GRB detection.pdf

5 Discussion

16:00 Coffee Break

Session 2: Data Science and Astrophysics

Convener: Chair: Vito Conforti (Istituto Nazionale di Astrofisica (INAF))

6 Improving accuracy of Machine Learning algorithms with Feature Selection in the context of particle background discrimination for the AGILE telescope

The aim of this work is to optimise the training process of machine learning algorithms in order to improve their performance on the background particles discrimination task, for the AGILE gamma ray telescope. It has been used a supervised dataset that describes the interactions between the high energy particles with the AGILE detectors. The dataset is composed of thousands of particles interactions, each interaction is described by more than 250 features. The optimisation of the training process is carried out through the Feature Selection technique: the application of selection/dimensionality reduction methods to extract the most relevant features for the training phase of the machine learning algorithms. This process can lead to improvements in their accuracy scores.

Speaker: Leonardo Baroncelli (Istituto Nazionale di Astrofisica (INAF))

Deep Learning @ INAF_baroncelli.pdf

7 Discovery of new QSOs by means of Canonical Correlation Analysis and photometric catalogs.

The several astronomical surveys carried on in the past, and those currently ongoing, produced a huge amount of data stored in their legacy catalogs. Besides the primary scientific goals pursued by the individual experiments, there probably is a lot of information hidden in those databases, still waiting to be extracted. This task is of crucial importance both because it provides further knowledge without requiring new instruments/experiments, and (most importantly) because the experience acquired in digging up legacy databases will allow us to properly exploit the overwhelming volume of data coming from the astronomical facilities in the next decade.

In this talk I will present a new method to discover new, high-redshift (z > 2.5) QSOs using Canonical Correlation Analysis on photometric catalogs in the optical and infrared wavelengths. We focused our attention on the Southern Hemisphere, since (historically) dedicated QSO observations have been carried out mainly in the North, and took advantage of the
recent Skymapper survey, being carried on from the Siding Spring Observatory in Australia.

We cross-correlated the Skymapper photometric catalog with the GAIA, 2MASS and WISE databases and applied our method in order to find new QSOs among the sources without an object type classification. We also carried on a few observations at La Silla and Las Campanas Observatories in order to test our predictions, and discovered 119 new, bright (i<18), QSOs with z in the range 2.5 - 4.89. The method proved to be very efficient, with a success rate of new QSOs identification of ~70%, and an estimated completeness of ~90%.

Speaker: Giorgio Calderone (Istituto Nazionale di Astrofisica (INAF))

8 A real time approach targeted at buyer prediction, using behaviour classification in Tourism Web Analytics

Given the continuously growing importance of digital marketing in the tourism sector, understanding customer’s purchase behaviour is a critical issue for the online competition. Competitors who are able to identify prospective patterns can create a value to lift business.

This work presents a supervised machine-learning (ML) approach, focused on the buyer prediction, in the context of online behavioural targeting to analyze in real time the user session, and predicting precisely if she/he would buy or not.

The appropriate selection of data samples is important for effective analysis and prediction based on behavioural patterns, for this purpose as use case is used a huge dataset representing the user experiences in a hotel/holydays structure to train and test an ML model. The dataset is organized in 669.653 user sessions defined as observation that can be brought back to users that finalize a purchase (positives) and user session that not purchase (negatives). The data analysis highlighted the great unbalancing of the observations as positives (4%) versus negatives (96%) that is solved by applying the Under Sampling (Scikit-Learn) algorithm to obtain a leveled dataset to train the model. A Feature engineering analysis is done by selecting the best features that produce less errors and best accuracy. Our training method is based on a hybrid combination of the Gradient Boost Classifier (XGBoost and LightGBM) and Decision Tree Classifier (Scikit-Learn) to perform the binary classification.

We performed our experiments developing a prediction module as a RESTFull service, that the has been connected in the retail application. When a user session is collected in real-time, the module make a prediction that can be evaluated to put in place some action to engage the user. The evaluation experiment of the RESTFull service in production environment took place over a period of six months and the collected data were further analyzed. During the experiment ~240.000 real-time predictions were generated in total, 5% of these are positive predictions of which 87% are correct (accuracy > 70%), 95% of these are negative predictions of which 80% are correct (accuracy > 70%).

Our suggested approach compared to the last six months, before the start of the test, seems capable of dealing with more complex online advertising models. In particular we evaluate the impact in the marketing business noting a 70% decrease of management time and 90% increase in viewability of the proposed product to sell.

The obtained results are promising and encourage us to continue experimentation with more sophisticated models or other algorithms to further improve the performance of the system.
In the next steps we are planning experiments to improve the model prediction with more information obtained from third party data providers (eg. work-calendar, national-holidays, events...) and than introducing the temporal dimension to our model to apply time series analysis techniques.

Speaker: Massimo Deriu (CRS4)

CRS4 - Presentazione INAF.pdf

9 Discussion

Thursday 19 September

Session 3: Commercial tools & Cloud

Convener: Chair: Marco Landoni (Istituto Nazionale di Astrofisica (INAF))

10 IBM deep learning

Speaker: Eleonora Picca (IBM)

IBM_Deep_learning_INAF_shared.pdf

11 Amazon Web Services - SageMaker platform

This talk will illustrate the SageMaker platform from AWS, a Cloud Based MLaaS service that improve, speed up and enhance the deployment and training of machine learning algorithms. The session will be live, with a full End-to-end example performed on live screen with the AWS console. This aims at training people on AWS services, especially for the ones offered for machine learning.

Speaker: Federico d'Alessio (Amazon Web Services - Solution Architect)

11:00 Coffee Break

Session 3: Commercial tools & Cloud

Convener: Chair: Marco Landoni (Istituto Nazionale di Astrofisica (INAF))

12 Demystifying Deep Learning: A Practical Approach in MATLAB

Deep learning can achieve state-of-the-art accuracy in many humanlike tasks such as naming objects in a scene or recognizing optimal paths in an environment.

The main tasks are to assemble large data sets, create a neural network, to train, visualize, and evaluate different models, using specialized hardware - often requiring unique programming knowledge. These tasks are frequently even more challenging because of the complex theory behind them.

In this seminar, we’ll demonstrate new MATLAB features that simplify these tasks and eliminate the low-level programming. In doing so, we’ll decipher practical knowledge of the domain of deep learning. We’ll build and train neural networks that recognize images, classify signals, and figure out the interested area in an image at pixel level.

Along the way, you’ll see MATLAB features that make it easy to:

• Manage large sets of images
• Create, analyze, and visualize networks and gain insight into the black box nature of deep networks
• Build networks from scratch with a drag-and-drop interface
• Perform classification tasks on images and signals, and pixel-level semantic segmentation on images
• Use of models already available such as GoogLeNet and ResNet
• Import models from TensorFlow Keras, Caffe, and the ONNX Model format
• Speed up network training with parallel computing on a cluster
• Automate manual effort required to label ground truth
• Automatically generate source code for embedded targets

Speaker: Giuseppe Ridino (MathWorks)

Deep_Learning@INAF_with_MATLAB.pdf

Session 4: General discussion, in-depth insights by the audience with respect to commercial service proposals, practical examples of use

13:20 Lunch