Passed
Push — master ( 1ec1dd...430449 )
by Giacomo
09:28
created

hawc_hal.HAL   D

Complexity

Total Complexity 59

Size/Duplication

Total Lines 738
Duplicated Lines 0 %

Importance

Changes 0
Metric Value
wmc 59
eloc 356
dl 0
loc 738
rs 4.08
c 0
b 0
f 0

18 Methods

Rating   Name   Duplication   Size   Complexity  
B HAL.set_model() 0 44 5
A HAL.get_saturated_model_likelihood() 0 7 1
B HAL.__init__() 0 86 2
A HAL._compute_likelihood_biases() 0 16 2
C HAL.set_active_measurements() 0 35 9
A HAL._setup_psf_convolutors() 0 7 2
A HAL.display() 0 33 1
B HAL._get_expectation() 0 78 8
A HAL.write() 0 11 1
B HAL.display_spectrum() 0 92 3
B HAL.get_simulated_dataset() 0 62 7
A HAL.get_log_like() 0 37 3
A HAL._represent_healpix_map() 0 19 2
A HAL.inner_fit() 0 10 1
A HAL._get_optimal_xsize() 0 3 1
A HAL.display_stacked_image() 0 50 4
A HAL.get_number_of_data_points() 0 8 2
B HAL.display_fit() 0 92 5

How to fix   Complexity   

Complexity

Complex classes like hawc_hal.HAL often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

1
import collections
0 ignored issues
show
Coding Style introduced by
This module should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
Coding Style Naming introduced by
Module name "HAL" doesn't conform to snake_case naming style ('([a-z_][a-z0-9_]*)$' pattern)

This check looks for invalid names for a range of different identifiers.

You can set regular expressions to which the identifiers must conform if the defaults do not match your requirements.

If your project includes a Pylint configuration file, the settings contained in that file take precedence.

To find out more about Pylint, please refer to their site.

Loading history...
2
import numpy as np
0 ignored issues
show
introduced by
Unable to import 'numpy'
Loading history...
3
import healpy as hp
0 ignored issues
show
introduced by
Unable to import 'healpy'
Loading history...
4
import matplotlib.pyplot as plt
0 ignored issues
show
introduced by
Unable to import 'matplotlib.pyplot'
Loading history...
5
import copy
0 ignored issues
show
introduced by
standard import "import copy" should be placed before "import numpy as np"
Loading history...
6
from astropy.convolution import Gaussian2DKernel
0 ignored issues
show
introduced by
Unable to import 'astropy.convolution'
Loading history...
7
from astropy.convolution import convolve_fft as convolve
0 ignored issues
show
introduced by
Unable to import 'astropy.convolution'
Loading history...
8
from scipy.stats import poisson
0 ignored issues
show
introduced by
Unable to import 'scipy.stats'
Loading history...
9
10
from threeML.plugin_prototype import PluginPrototype
0 ignored issues
show
introduced by
Unable to import 'threeML.plugin_prototype'
Loading history...
11
from threeML.plugins.gammaln import logfactorial
0 ignored issues
show
introduced by
Unable to import 'threeML.plugins.gammaln'
Loading history...
12
from threeML.parallel import parallel_client
0 ignored issues
show
introduced by
Unable to import 'threeML.parallel'
Loading history...
13
from threeML.io.progress_bar import progress_bar
0 ignored issues
show
introduced by
Unable to import 'threeML.io.progress_bar'
Loading history...
14
15
from astromodels import Parameter
0 ignored issues
show
introduced by
Unable to import 'astromodels'
Loading history...
16
17
from maptree import map_tree_factory
0 ignored issues
show
Coding Style introduced by
Relative import 'maptree', should be 'hawc_hal.maptree'
Loading history...
18
from response import hawc_response_factory
0 ignored issues
show
Coding Style introduced by
Relative import 'response', should be 'hawc_hal.response'
Loading history...
19
from convolved_source import ConvolvedPointSource, \
0 ignored issues
show
Coding Style introduced by
Relative import 'convolved_source', should be 'hawc_hal.convolved_source'
Loading history...
20
    ConvolvedExtendedSource3D, ConvolvedExtendedSource2D, ConvolvedSourcesContainer
21
from healpix_handling import FlatSkyToHealpixTransform
0 ignored issues
show
Coding Style introduced by
Relative import 'healpix_handling', should be 'hawc_hal.healpix_handling'
Loading history...
22
from healpix_handling import SparseHealpix
0 ignored issues
show
Coding Style introduced by
Relative import 'healpix_handling', should be 'hawc_hal.healpix_handling'
Loading history...
23
from healpix_handling import get_gnomonic_projection
0 ignored issues
show
Coding Style introduced by
Relative import 'healpix_handling', should be 'hawc_hal.healpix_handling'
Loading history...
24
from psf_fast import PSFConvolutor
0 ignored issues
show
Coding Style introduced by
Relative import 'psf_fast', should be 'hawc_hal.psf_fast'
Loading history...
25
from log_likelihood import log_likelihood
0 ignored issues
show
Coding Style introduced by
Relative import 'log_likelihood', should be 'hawc_hal.log_likelihood'
Loading history...
26
from util import ra_to_longitude
0 ignored issues
show
Coding Style introduced by
Relative import 'util', should be 'hawc_hal.util'
Loading history...
27
28
29
class HAL(PluginPrototype):
0 ignored issues
show
Coding Style introduced by
This class should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
best-practice introduced by
Too many instance attributes (16/7)
Loading history...
30
31
    def __init__(self, name, maptree, response_file, roi, flat_sky_pixels_sizes=0.17):
0 ignored issues
show
best-practice introduced by
Too many arguments (6/5)
Loading history...
32
33
        # Store ROI
34
        self._roi = roi
35
36
        # Set up the flat-sky projection
37
38
        self._flat_sky_projection = roi.get_flat_sky_projection(flat_sky_pixels_sizes)
39
40
        # Read map tree (data)
41
42
        self._maptree = map_tree_factory(maptree, roi=roi)
43
44
        # Read detector response_file
45
46
        self._response = hawc_response_factory(response_file)
47
48
        # Use a renormalization of the background as nuisance parameter
49
        # NOTE: it is fixed to 1.0 unless the user explicitly sets it free (experimental)
50
        self._nuisance_parameters = collections.OrderedDict()
51
        self._nuisance_parameters['%s_bkg_renorm' % name] = Parameter('%s_bkg_renorm' % name, 1.0,
52
                                                                      min_value=0.5, max_value=1.5,
53
                                                                      delta=0.01,
54
                                                                      desc="Renormalization for background map",
55
                                                                      free=False,
56
                                                                      is_normalization=False)
57
58
        # Instance parent class
59
60
        super(HAL, self).__init__(name, self._nuisance_parameters)
61
62
        self._likelihood_model = None
63
64
        # These lists will contain the maps for the point sources
65
        self._convolved_point_sources = ConvolvedSourcesContainer()
66
        # and this one for extended sources
67
        self._convolved_ext_sources = ConvolvedSourcesContainer()
68
69
        # All energy/nHit bins are loaded in memory
70
        self._all_planes = list(self._maptree.analysis_bins_labels)
71
72
        # The active planes list always contains the list of *indexes* of the active planes
73
        self._active_planes = None
74
75
        # Set up the transformations from the flat-sky projection to Healpix, as well as the list of active pixels
76
        # (one for each energy/nHit bin). We make a separate transformation because different energy bins might have
77
        # different nsides
78
        self._active_pixels = collections.OrderedDict()
79
        self._flat_sky_to_healpix_transform = collections.OrderedDict()
80
81
        for bin_id in self._maptree:
82
83
            this_maptree = self._maptree[bin_id]
84
            this_nside = this_maptree.nside
85
            this_active_pixels = roi.active_pixels(this_nside)
86
87
            this_flat_sky_to_hpx_transform = FlatSkyToHealpixTransform(self._flat_sky_projection.wcs,
88
                                                                       'icrs',
89
                                                                       this_nside,
90
                                                                       this_active_pixels,
91
                                                                       (self._flat_sky_projection.npix_width,
92
                                                                        self._flat_sky_projection.npix_height),
93
                                                                       order='bilinear')
94
95
            self._active_pixels[bin_id] = this_active_pixels
96
            self._flat_sky_to_healpix_transform[bin_id] = this_flat_sky_to_hpx_transform
97
98
        # This will contain a list of PSF convolutors for extended sources, if there is any in the model
99
100
        self._psf_convolutors = None
101
102
        # Pre-compute the log-factorial factor in the likelihood, so we do not keep to computing it over and over
103
        # again.
104
        self._log_factorials = collections.OrderedDict()
105
106
        # We also apply a bias so that the numerical value of the log-likelihood stays small. This helps when
107
        # fitting with algorithms like MINUIT because the convergence criterium involves the difference between
108
        # two likelihood values, which would be affected by numerical precision errors if the two values are
109
        # too large
110
        self._saturated_model_like_per_maptree = collections.OrderedDict()
111
112
        # The actual computation is in a method so we can recall it on clone (see the get_simulated_dataset method)
113
        self._compute_likelihood_biases()
114
115
        # This will save a clone of self for simulations
116
        self._clone = None
117
118
    def _setup_psf_convolutors(self):
119
120
        central_response_bins = self._response.get_response_dec_bin(self._roi.ra_dec_center[1])
121
122
        self._psf_convolutors = collections.OrderedDict()
123
        for bin_id in central_response_bins:
124
            self._psf_convolutors[bin_id] = PSFConvolutor(central_response_bins[bin_id].psf, self._flat_sky_projection)
125
126
    def _compute_likelihood_biases(self):
127
128
        for bin_label in self._maptree:
129
130
            data_analysis_bin = self._maptree[bin_label]
131
132
            this_log_factorial = np.sum(logfactorial(data_analysis_bin.observation_map.as_partial()))
133
            self._log_factorials[bin_label] = this_log_factorial
134
135
            # As bias we use the likelihood value for the saturated model
136
            obs = data_analysis_bin.observation_map.as_partial()
137
            bkg = data_analysis_bin.background_map.as_partial()
138
139
            sat_model = np.clip(obs - bkg, 1e-50, None).astype(np.float64)
140
141
            self._saturated_model_like_per_maptree[bin_label] = log_likelihood(obs, bkg, sat_model) - this_log_factorial
142
143
    def get_saturated_model_likelihood(self):
144
        """
145
        Returns the likelihood for the saturated model (i.e. a model exactly equal to observation - background).
146
147
        :return:
148
        """
149
        return sum(self._saturated_model_like_per_maptree.values())
150
151
    def set_active_measurements(self, bin_id_min=None, bin_id_max=None, bin_list=None):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
152
153
        # Check for legal input
154
        if bin_id_min is not None:
155
156
            assert bin_id_max is not None, "If you provide a minimum bin, you also need to provide a maximum bin"
157
158
            # Make sure they are integers
159
            bin_id_min = int(bin_id_min)
160
            bin_id_max = int(bin_id_max)
161
162
            self._active_planes = []
163
            for this_bin in range(bin_id_min, bin_id_max + 1):
164
                this_bin = str(this_bin)
165
                if this_bin not in self._all_planes:
166
167
                    raise ValueError("Bin %s it not contained in this response" % this_bin)
168
169
                self._active_planes.append(this_bin)
170
171
        else:
172
173
            assert bin_id_max is None, "If you provide a maximum bin, you also need to provide a minimum bin"
174
175
            assert bin_list is not None
176
177
            self._active_planes = []
178
179
            for this_bin in bin_list:
180
181
                if not this_bin in self._all_planes:
182
183
                    raise ValueError("Bin %s it not contained in this response" % this_bin)
184
185
                self._active_planes.append(this_bin)
186
187
    def display(self, verbose=False):
188
        """
189
        Prints summary of the current object content.
190
        """
191
192
        print("Region of Interest: ")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
193
        print("--------------------\n")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
194
        self._roi.display()
195
196
        print("")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
197
        print("Flat sky projection: ")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
198
        print("----------------------\n")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
199
200
        print("Width x height: %s x %s px" % (self._flat_sky_projection.npix_width,
201
                                              self._flat_sky_projection.npix_height))
202
        print("Pixel sizes: %s deg" % self._flat_sky_projection.pixel_size)
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
203
204
        print("")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
205
        print("Response: ")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
206
        print("----------\n")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
207
208
        self._response.display(verbose)
209
210
        print("")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
211
        print("Map Tree: ")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
212
        print("----------\n")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
213
214
        self._maptree.display()
215
216
        print("")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
217
        print("Active energy/nHit planes: ")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
218
        print("---------------------------\n")
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
219
        print(self._active_planes)
0 ignored issues
show
Unused Code Coding Style introduced by
There is an unnecessary parenthesis after print.
Loading history...
220
221
    def set_model(self, likelihood_model_instance):
222
        """
223
        Set the model to be used in the joint minimization. Must be a LikelihoodModel instance.
224
        """
225
226
        self._likelihood_model = likelihood_model_instance
227
228
        # Reset
229
        self._convolved_point_sources.reset()
230
        self._convolved_ext_sources.reset()
231
232
        # For each point source in the model, build the convolution class
233
234
        for source in self._likelihood_model.point_sources.values():
235
236
            this_convolved_point_source = ConvolvedPointSource(source, self._response, self._flat_sky_projection)
237
238
            self._convolved_point_sources.append(this_convolved_point_source)
239
240
        # Samewise for extended sources
241
242
        ext_sources = self._likelihood_model.extended_sources.values()
243
244
        if len(ext_sources) > 0:
0 ignored issues
show
Unused Code introduced by
Do not use len(SEQUENCE) to determine if a sequence is empty
Loading history...
245
246
            # We will need to convolve
247
248
            self._setup_psf_convolutors()
249
250
            for source in ext_sources:
251
252
                if source.spatial_shape.n_dim == 2:
253
254
                    this_convolved_ext_source = ConvolvedExtendedSource2D(source,
255
                                                                          self._response,
256
                                                                          self._flat_sky_projection)
257
258
                else:
259
260
                    this_convolved_ext_source = ConvolvedExtendedSource3D(source,
261
                                                                          self._response,
262
                                                                          self._flat_sky_projection)
263
264
                self._convolved_ext_sources.append(this_convolved_ext_source)
265
266
    def display_spectrum(self):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
Comprehensibility introduced by
This function exceeds the maximum number of variables (26/15).
Loading history...
267
268
        n_point_sources = self._likelihood_model.get_number_of_point_sources()
269
        n_ext_sources = self._likelihood_model.get_number_of_extended_sources()
270
271
        total_counts = np.zeros(len(self._active_planes), dtype=float)
272
        total_model = np.zeros_like(total_counts)
273
        model_only = np.zeros_like(total_counts)
274
        net_counts = np.zeros_like(total_counts)
275
        yerr_low = np.zeros_like(total_counts)
276
        yerr_high = np.zeros_like(total_counts)
277
278
        for i, energy_id in enumerate(self._active_planes):
279
280
            data_analysis_bin = self._maptree[energy_id]
281
282
            this_model_map_hpx = self._get_expectation(data_analysis_bin, energy_id, n_point_sources, n_ext_sources)
283
284
            this_model_tot = np.sum(this_model_map_hpx)
285
286
            this_data_tot = np.sum(data_analysis_bin.observation_map.as_partial())
287
            this_bkg_tot = np.sum(data_analysis_bin.background_map.as_partial())
288
289
            total_counts[i] = this_data_tot
290
            net_counts[i] = this_data_tot - this_bkg_tot
291
            model_only[i] = this_model_tot
292
293
            this_wh_model = this_model_tot + this_bkg_tot
294
            total_model[i] = this_wh_model
295
296
            if this_data_tot >= 50.0:
297
298
                # Gaussian limit
299
                # Under the null hypothesis the data are distributed as a Gaussian with mu = model
300
                # and sigma = sqrt(model)
301
                # NOTE: since we neglect the background uncertainty, the background is part of the
302
                # model
303
                yerr_low[i] = np.sqrt(this_data_tot)
304
                yerr_high[i] = np.sqrt(this_data_tot)
305
306
            else:
307
308
                # Low-counts
309
                # Under the null hypothesis the data are distributed as a Poisson distribution with
310
                # mean = model, plot the 68% confidence interval (quantile=[0.16,1-0.16]).
311
                # NOTE: since we neglect the background uncertainty, the background is part of the
312
                # model
313
                quantile = 0.16
314
                mean = this_wh_model
315
                y_low = poisson.isf(1-quantile, mu=mean)
316
                y_high = poisson.isf(quantile, mu=mean)
317
                yerr_low[i] = mean-y_low
318
                yerr_high[i] = y_high-mean
319
320
        residuals = (total_counts - total_model) / np.sqrt(total_model)
321
        residuals_err = [yerr_high / np.sqrt(total_model),
322
                         yerr_low / np.sqrt(total_model)]
323
324
        fig, subs = plt.subplots(2, 1, gridspec_kw={'height_ratios': [2, 1], 'hspace': 0})
325
326
        subs[0].errorbar(self._active_planes, net_counts, yerr=[yerr_high, yerr_low],
327
                         capsize=0,
328
                         color='black', label='Net counts', fmt='.')
329
330
        subs[0].plot(self._active_planes, model_only, label='Convolved model')
331
332
        subs[0].legend(bbox_to_anchor=(1.0, 1.0), loc="upper right",
333
                       numpoints=1)
334
335
        # Residuals
336
        subs[1].axhline(0, linestyle='--')
337
338
        subs[1].errorbar(
339
            self._active_planes, residuals,
340
            yerr=residuals_err,
341
            capsize=0, fmt='.'
342
        )
343
344
        y_limits = [min(net_counts[net_counts > 0]) / 2., max(net_counts) * 2.]
345
346
        subs[0].set_yscale("log", nonposy='clip')
347
        subs[0].set_ylabel("Counts per bin")
348
        subs[0].set_xticks([])
349
350
        subs[1].set_xlabel("Analysis bin")
351
        subs[1].set_ylabel(r"$\frac{{cts - mod - bkg}}{\sqrt{mod + bkg}}$")
352
        subs[1].set_xticks(self._active_planes)
353
        subs[1].set_xticklabels(self._active_planes)
354
355
        subs[0].set_ylim(y_limits)
356
357
        return fig
358
359
    def get_log_like(self):
360
        """
361
        Return the value of the log-likelihood with the current values for the
362
        parameters
363
        """
364
365
        n_point_sources = self._likelihood_model.get_number_of_point_sources()
366
        n_ext_sources = self._likelihood_model.get_number_of_extended_sources()
367
368
        # Make sure that no source has been added since we filled the cache
369
        assert n_point_sources == self._convolved_point_sources.n_sources_in_cache and \
370
               n_ext_sources == self._convolved_ext_sources.n_sources_in_cache, \
371
            "The number of sources has changed. Please re-assign the model to the plugin."
372
373
        # This will hold the total log-likelihood
374
        total_log_like = 0
375
376
        for bin_id in self._active_planes:
377
378
            data_analysis_bin = self._maptree[bin_id]
379
380
            this_model_map_hpx = self._get_expectation(data_analysis_bin, bin_id, n_point_sources, n_ext_sources)
381
382
            # Now compare with observation
383
            bkg_renorm = self._nuisance_parameters.values()[0].value
384
385
            obs = data_analysis_bin.observation_map.as_partial()  # type: np.array
386
            bkg = data_analysis_bin.background_map.as_partial() * bkg_renorm  # type: np.array
387
388
            this_pseudo_log_like = log_likelihood(obs,
389
                                                  bkg,
390
                                                  this_model_map_hpx)
391
392
            total_log_like += this_pseudo_log_like - self._log_factorials[bin_id] \
393
                              - self._saturated_model_like_per_maptree[bin_id]
394
395
        return total_log_like
396
397
    def write(self, response_file_name, map_tree_file_name):
398
        """
399
        Write this dataset to disk in HDF format.
400
401
        :param response_file_name: filename for the response
402
        :param map_tree_file_name: filename for the map tree
403
        :return: None
404
        """
405
406
        self._maptree.write(map_tree_file_name)
407
        self._response.write(response_file_name)
408
409
    def get_simulated_dataset(self, name):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
410
411
        # First get expectation under the current model and store them, if we didn't do it yet
412
413
        if self._clone is None:
414
415
            n_point_sources = self._likelihood_model.get_number_of_point_sources()
416
            n_ext_sources = self._likelihood_model.get_number_of_extended_sources()
417
418
            expectations = []
419
420
            for i, data_analysis_bin in enumerate(self._maptree):
421
422
                if i not in self._active_planes:
423
424
                    expectations.append(None)
425
426
                else:
427
428
                    expectations.append(self._get_expectation(data_analysis_bin, i,
429
                                                              n_point_sources, n_ext_sources) +
430
                                        data_analysis_bin.background_map.as_partial())
431
432
            if parallel_client.is_parallel_computation_active():
433
434
                # Do not clone, as the parallel environment already makes clones
435
436
                clone = self
437
438
            else:
439
440
                clone = copy.deepcopy(self)
441
442
            self._clone = (clone, expectations)
443
444
        # Substitute the observation and background for each data analysis bin
445
        for i, (data_analysis_bin, orig_data_analysis_bin) in enumerate(zip(self._clone[0]._maptree, self._maptree)):
0 ignored issues
show
Coding Style Best Practice introduced by
It seems like _maptree was declared protected and should not be accessed from this context.

Prefixing a member variable _ is usually regarded as the equivalent of declaring it with protected visibility that exists in other languages. Consequentially, such a member should only be accessed from the same class or a child class:

class MyParent:
    def __init__(self):
        self._x = 1;
        self.y = 2;

class MyChild(MyParent):
    def some_method(self):
        return self._x    # Ok, since accessed from a child class

class AnotherClass:
    def some_method(self, instance_of_my_child):
        return instance_of_my_child._x   # Would be flagged as AnotherClass is not
                                         # a child class of MyParent
Loading history...
Unused Code introduced by
The variable orig_data_analysis_bin seems to be unused.
Loading history...
446
447
            if i not in self._active_planes:
448
449
                continue
450
451
            else:
452
453
                # Active plane. Generate new data
454
                expectation = self._clone[1][i]
455
                new_data = np.random.poisson(expectation, size=(1, expectation.shape[0])).flatten()
456
457
                # Substitute data
458
                data_analysis_bin.observation_map.set_new_values(new_data)
459
460
        # Now change name and return
461
        self._clone[0]._name = name
0 ignored issues
show
Coding Style Best Practice introduced by
It seems like _name was declared protected and should not be accessed from this context.

Prefixing a member variable _ is usually regarded as the equivalent of declaring it with protected visibility that exists in other languages. Consequentially, such a member should only be accessed from the same class or a child class:

class MyParent:
    def __init__(self):
        self._x = 1;
        self.y = 2;

class MyChild(MyParent):
    def some_method(self):
        return self._x    # Ok, since accessed from a child class

class AnotherClass:
    def some_method(self, instance_of_my_child):
        return instance_of_my_child._x   # Would be flagged as AnotherClass is not
                                         # a child class of MyParent
Loading history...
462
        # Adjust the name of the nuisance parameter
463
        old_name = self._clone[0]._nuisance_parameters.keys()[0]
0 ignored issues
show
Coding Style Best Practice introduced by
It seems like _nuisance_parameters was declared protected and should not be accessed from this context.

Prefixing a member variable _ is usually regarded as the equivalent of declaring it with protected visibility that exists in other languages. Consequentially, such a member should only be accessed from the same class or a child class:

class MyParent:
    def __init__(self):
        self._x = 1;
        self.y = 2;

class MyChild(MyParent):
    def some_method(self):
        return self._x    # Ok, since accessed from a child class

class AnotherClass:
    def some_method(self, instance_of_my_child):
        return instance_of_my_child._x   # Would be flagged as AnotherClass is not
                                         # a child class of MyParent
Loading history...
464
        new_name = old_name.replace(self.name, name)
465
        self._clone[0]._nuisance_parameters[new_name] = self._clone[0]._nuisance_parameters.pop(old_name)
0 ignored issues
show
Coding Style Best Practice introduced by
It seems like _nuisance_parameters was declared protected and should not be accessed from this context.

Prefixing a member variable _ is usually regarded as the equivalent of declaring it with protected visibility that exists in other languages. Consequentially, such a member should only be accessed from the same class or a child class:

class MyParent:
    def __init__(self):
        self._x = 1;
        self.y = 2;

class MyChild(MyParent):
    def some_method(self):
        return self._x    # Ok, since accessed from a child class

class AnotherClass:
    def some_method(self, instance_of_my_child):
        return instance_of_my_child._x   # Would be flagged as AnotherClass is not
                                         # a child class of MyParent
Loading history...
466
467
        # Recompute biases
468
        self._clone[0]._compute_likelihood_biases()
0 ignored issues
show
Coding Style Best Practice introduced by
It seems like _compute_likelihood_biases was declared protected and should not be accessed from this context.

Prefixing a member variable _ is usually regarded as the equivalent of declaring it with protected visibility that exists in other languages. Consequentially, such a member should only be accessed from the same class or a child class:

class MyParent:
    def __init__(self):
        self._x = 1;
        self.y = 2;

class MyChild(MyParent):
    def some_method(self):
        return self._x    # Ok, since accessed from a child class

class AnotherClass:
    def some_method(self, instance_of_my_child):
        return instance_of_my_child._x   # Would be flagged as AnotherClass is not
                                         # a child class of MyParent
Loading history...
469
470
        return self._clone[0]
471
472
    def _get_expectation(self, data_analysis_bin, energy_bin_id, n_point_sources, n_ext_sources):
473
474
        # Compute the expectation from the model
475
476
        this_model_map = None
477
478
        for pts_id in range(n_point_sources):
479
480
            this_convolved_source = self._convolved_point_sources[pts_id]
481
482
            expectation_per_transit = this_convolved_source.get_source_map(energy_bin_id, tag=None)
483
484
            expectation_from_this_source = expectation_per_transit * data_analysis_bin.n_transits
485
486
            if this_model_map is None:
487
488
                # First addition
489
490
                this_model_map = expectation_from_this_source
491
492
            else:
493
494
                this_model_map += expectation_from_this_source
495
496
        # Now process extended sources
497
        if n_ext_sources > 0:
498
499
            this_ext_model_map = None
500
501
            for ext_id in range(n_ext_sources):
502
503
                this_convolved_source = self._convolved_ext_sources[ext_id]
504
505
                expectation_per_transit = this_convolved_source.get_source_map(energy_bin_id)
506
507
                if this_ext_model_map is None:
508
509
                    # First addition
510
511
                    this_ext_model_map = expectation_per_transit
512
513
                else:
514
515
                    this_ext_model_map += expectation_per_transit
516
517
            # Now convolve with the PSF
518
            if this_model_map is None:
519
                
0 ignored issues
show
Coding Style introduced by
Trailing whitespace
Loading history...
520
                # Only extended sources
521
            
0 ignored issues
show
Coding Style introduced by
Trailing whitespace
Loading history...
522
                this_model_map = (self._psf_convolutors[energy_bin_id].extended_source_image(this_ext_model_map) *
523
                                  data_analysis_bin.n_transits)
524
            
0 ignored issues
show
Coding Style introduced by
Trailing whitespace
Loading history...
525
            else:
526
527
                this_model_map += (self._psf_convolutors[energy_bin_id].extended_source_image(this_ext_model_map) *
528
                                   data_analysis_bin.n_transits)
529
530
531
        # Now transform from the flat sky projection to HEALPiX
532
533
        if this_model_map is not None:
534
535
            # First divide for the pixel area because we need to interpolate brightness
536
            this_model_map = this_model_map / self._flat_sky_projection.project_plane_pixel_area
537
538
            this_model_map_hpx = self._flat_sky_to_healpix_transform[energy_bin_id](this_model_map, fill_value=0.0)
539
540
            # Now multiply by the pixel area of the new map to go back to flux
541
            this_model_map_hpx *= hp.nside2pixarea(data_analysis_bin.nside, degrees=True)
542
543
        else:
544
545
            # No sources
546
547
            this_model_map_hpx = 0.0
548
549
        return this_model_map_hpx
550
551
    @staticmethod
552
    def _represent_healpix_map(fig, hpx_map, longitude, latitude, xsize, resolution, smoothing_kernel_sigma):
0 ignored issues
show
best-practice introduced by
Too many arguments (7/5)
Loading history...
553
554
        proj = get_gnomonic_projection(fig, hpx_map,
555
                                       rot=(longitude, latitude, 0.0),
556
                                       xsize=xsize,
557
                                       reso=resolution)
558
559
        if smoothing_kernel_sigma is not None:
560
561
            # Get the sigma in pixels
562
            sigma = smoothing_kernel_sigma * 60 / resolution
563
564
            proj = convolve(list(proj),
565
                            Gaussian2DKernel(sigma),
566
                            nan_treatment='fill',
567
                            preserve_nan=True)
568
569
        return proj
570
571
    def display_fit(self, smoothing_kernel_sigma=0.1, display_colorbar=False):
0 ignored issues
show
Comprehensibility introduced by
This function exceeds the maximum number of variables (28/15).
Loading history...
572
        """
573
        Display the model, excess and residual maps for each analysis bin (plane).
574
        3 columns, len(self._active_planes) lines.
575
        """
576
577
        n_point_sources = self._likelihood_model.get_number_of_point_sources()
578
        n_ext_sources = self._likelihood_model.get_number_of_extended_sources()
579
580
        # This is the resolution (i.e., the size of one pixel) of the image
581
        resolution = 3.0 # arcmin
582
583
        # The image is going to cover the diameter plus 20% padding
584
        xsize = self._get_optimal_xsize(resolution)
585
586
        n_active_planes = len(self._active_planes)
587
588
        fig, subs = plt.subplots(n_active_planes, 3, figsize=(8, n_active_planes * 2))
589
590
        with progress_bar(len(self._active_planes), title='Smoothing maps') as prog_bar:
591
592
            images = ['None'] * 3
593
594
            for i, plane_id in enumerate(self._active_planes):
595
596
                data_analysis_bin = self._maptree[plane_id]
597
598
                # Get the center of the projection for this plane
599
                this_ra, this_dec = self._roi.ra_dec_center
600
601
                this_model_map_hpx = self._get_expectation(data_analysis_bin, plane_id, n_point_sources, n_ext_sources)
602
603
                # Make a full healpix map for a second
604
                whole_map = SparseHealpix(this_model_map_hpx,
605
                                          self._active_pixels[plane_id],
606
                                          data_analysis_bin.observation_map.nside).as_dense()
607
608
                # Healpix uses longitude between -180 and 180, while R.A. is between 0 and 360. We need to fix that:
609
                longitude = ra_to_longitude(this_ra)
610
611
                # Declination is already between -90 and 90
612
                latitude = this_dec
613
614
                # Plot model
615
616
                proj_m = self._represent_healpix_map(fig, whole_map,
617
                                                     longitude, latitude,
618
                                                     xsize, resolution, smoothing_kernel_sigma)
619
620
                images[0] = subs[i][0].imshow(proj_m, origin='lower')
621
622
                # Remove numbers from axis
623
                subs[i][0].axis('off')
624
625
                # Plot data map
626
                # Here we removed the background otherwise nothing is visible
627
                # Get background (which is in a way "part of the model" since the uncertainties are neglected)
628
                background_map = data_analysis_bin.background_map.as_dense()
629
                bkg_subtracted = data_analysis_bin.observation_map.as_dense() - background_map
630
631
                proj_d = self._represent_healpix_map(fig, bkg_subtracted,
632
                                                     longitude, latitude,
633
                                                     xsize, resolution, smoothing_kernel_sigma)
634
635
                images[1] = subs[i][1].imshow(proj_d, origin='lower')
636
637
                # Remove numbers from axis
638
                subs[i][1].axis('off')
639
640
                # Now residuals
641
                res = proj_d - proj_m
642
                # proj_res = self._represent_healpix_map(fig, res,
643
                #                                        longitude, latitude,
644
                #                                        xsize, resolution, smoothing_kernel_sigma)
645
                images[2] = subs[i][2].imshow(res, origin='lower')
646
647
                # Remove numbers from axis
648
                subs[i][2].axis('off')
649
650
                subs[i][0].set_title('model, bin {}'.format(data_analysis_bin.name))
651
                subs[i][1].set_title('excess, bin {}'.format(data_analysis_bin.name))
652
                subs[i][2].set_title('residuals, bin {}'.format(data_analysis_bin.name))
653
654
                if display_colorbar:
655
                    for j, image in enumerate(images):
656
                        plt.colorbar(image, ax=subs[i][j])
657
658
                prog_bar.increase()
659
660
        fig.set_tight_layout(True)
661
662
        return fig
663
664
    def _get_optimal_xsize(self, resolution):
665
666
        return 2.2 * self._roi.data_radius.to("deg").value / (resolution / 60.0)
667
668
    def display_stacked_image(self, smoothing_kernel_sigma=0.5):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
Comprehensibility introduced by
This function exceeds the maximum number of variables (19/15).
Loading history...
669
670
        # This is the resolution (i.e., the size of one pixel) of the image in arcmin
671
        resolution = 3.0
672
673
        # The image is going to cover the diameter plus 20% padding
674
        xsize = self._get_optimal_xsize(resolution)
675
676
        active_planes_bins = map(lambda x: self._maptree[x], self._active_planes)
0 ignored issues
show
introduced by
map/filter on lambda could be replaced by comprehension
Loading history...
677
678
        # Get the center of the projection for this plane
679
        this_ra, this_dec = self._roi.ra_dec_center
680
681
        # Healpix uses longitude between -180 and 180, while R.A. is between 0 and 360. We need to fix that:
682
        longitude = ra_to_longitude(this_ra)
683
684
        # Declination is already between -90 and 90
685
        latitude = this_dec
686
687
        total = None
688
689
        for i, data_analysis_bin in enumerate(active_planes_bins):
690
691
            # Plot data
692
            background_map = data_analysis_bin.background_map.as_dense()
693
            this_data = data_analysis_bin.observation_map.as_dense() - background_map
694
            idx = np.isnan(this_data)
695
            # this_data[idx] = hp.UNSEEN
696
697
            if i == 0:
698
699
                total = this_data
700
701
            else:
702
703
                # Sum only when there is no UNSEEN, so that the UNSEEN pixels will stay UNSEEN
704
                total[~idx] += this_data[~idx]
705
706
        delta_coord = (self._roi.data_radius.to("deg").value * 2.0) / 15.0
707
708
        fig, sub = plt.subplots(1,1)
0 ignored issues
show
Coding Style introduced by
Exactly one space required after comma
Loading history...
709
710
        proj = self._represent_healpix_map(fig, total, longitude, latitude, xsize, resolution, smoothing_kernel_sigma)
711
712
        sub.imshow(proj, origin='lower')
713
        sub.axis('off')
714
715
        hp.graticule(delta_coord, delta_coord)
716
717
        return fig
718
719
    def inner_fit(self):
720
        """
721
        This is used for the profile likelihood. Keeping fixed all parameters in the
722
        LikelihoodModel, this method minimize the logLike over the remaining nuisance
723
        parameters, i.e., the parameters belonging only to the model for this
724
        particular detector. If there are no nuisance parameters, simply return the
725
        logLike value.
726
        """
727
728
        return self.get_log_like()
729
730
    def get_number_of_data_points(self):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
731
732
        n_points = 0
733
734
        for bin_id in self._maptree:
735
            n_points += self._maptree[bin_id].observation_map.as_partial().shape[0]
736
737
        return n_points
738