Completed
Push — master ( a0f60d...4d37c6 )
by Andrei
02:03
created

cnn_dynamic.__allocate_neuron_patterns()   A

Complexity

Conditions 3

Size

Total Lines 18

Duplication

Lines 18
Ratio 100 %

Importance

Changes 0
Metric Value
cc 3
c 0
b 0
f 0
dl 18
loc 18
rs 9.4285
1
"""!
2
3
@brief Chaotic Neural Network
4
@details Based on article description:
5
         - E.N.Benderskaya, S.V.Zhukova. Large-dimension image clustering by means of fragmentary synchronization in chaotic systems. 2007.
6
         - E.N.Benderskaya, S.V.Zhukova. Clustering by Chaotic Neural Networks with Mean Field Calculated Via Delaunay Triangulation. 2008.
7
8
@authors Andrei Novikov ([email protected])
9
@date 2014-2016
10
@copyright GNU Public License
11
12
@cond GNU_PUBLIC_LICENSE
13
    PyClustering is free software: you can redistribute it and/or modify
14
    it under the terms of the GNU General Public License as published by
15
    the Free Software Foundation, either version 3 of the License, or
16
    (at your option) any later version.
17
    
18
    PyClustering is distributed in the hope that it will be useful,
19
    but WITHOUT ANY WARRANTY; without even the implied warranty of
20
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
21
    GNU General Public License for more details.
22
    
23
    You should have received a copy of the GNU General Public License
24
    along with this program.  If not, see <http://www.gnu.org/licenses/>.
25
@endcond
26
27
"""
28
29
import matplotlib.pyplot as plt;
0 ignored issues
show
Configuration introduced by
The import matplotlib.pyplot could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
30
import matplotlib.animation as animation;
0 ignored issues
show
Configuration introduced by
The import matplotlib.animation could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
Unused Code introduced by
Unused matplotlib.animation imported as animation
Loading history...
31
32
from matplotlib import rcParams;
0 ignored issues
show
Configuration introduced by
The import matplotlib could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
33
from matplotlib.font_manager import FontProperties;
0 ignored issues
show
Configuration introduced by
The import matplotlib.font_manager could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
34
35
import math;
36
import numpy;
0 ignored issues
show
Configuration introduced by
The import numpy could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
37
import random;
38
39
from enum import IntEnum;
40
41
from scipy.spatial import Delaunay;
0 ignored issues
show
Configuration introduced by
The import scipy.spatial could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
42
43
from pyclustering.utils import euclidean_distance_sqrt, average_neighbor_distance, heaviside, draw_dynamics;
44
45
46
class type_conn(IntEnum):
47
    """!
48
    @brief Enumeration of connection types for Chaotic Neural Network.
49
    
50
    @see cnn_network
51
    
52
    """
53
    
54
    ## All oscillators have connection with each other.
55
    ALL_TO_ALL  = 0,
56
    
57
    ## Connections between oscillators are created in line with Delaunay triangulation.
58
    TRIANGULATION_DELAUNAY = 1,
59
60
61
class cnn_dynamic:
62
    """!
63
    @brief Container of output dynamic of the chaotic neural network where states of each neuron during simulation are stored.
64
    
65
    @see cnn_network
66
    
67
    """
68
    
69
    def __init__(self, output = [], time = []):
0 ignored issues
show
Bug Best Practice introduced by
The default value [] might cause unintended side-effects.

Objects as default values are only created once in Python and not on each invocation of the function. If the default object is modified, this modification is carried over to the next invocation of the method.

# Bad:
# If array_param is modified inside the function, the next invocation will
# receive the modified object.
def some_function(array_param=[]):
    # ...

# Better: Create an array on each invocation
def some_function(array_param=None):
    array_param = array_param or []
    # ...
Loading history...
70
        """!
71
        @brief Costructor of the chaotic neural network output dynamic.
72
73
        @param[in] phase (list): Dynamic of oscillators on each step of simulation.
74
        @param[in] time (list): Simulation time.
75
        
76
        """
77
        
78
        ## Output value of each neuron on each iteration.
79
        self.output = output;
80
        
81
        ## Sequence of simulation steps of the network.
82
        self.time = time;
83
84
85
    def __len__(self):
86
        """!
87
        @brief (uint) Returns amount of simulation steps that are stored.
88
        
89
        """
90
        return len(self.output);
91
92
93 View Code Duplication
    def allocate_observation_matrix(self):
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated in your project.
Loading history...
94
        """!
95
        @brief Allocates observation matrix in line with output dynamic of the network.
96
        @details Matrix where state of each neuron is denoted by zero/one in line with Heaviside function on each iteration.
97
        
98
        @return (list) Observation matrix of the network dynamic.
99
        
100
        """
101
        number_neurons = len(self.output[0]);
102
        observation_matrix = [];
103
        
104
        for iteration in range(len(self.output)):
105
            obervation_column = [];
106
            for index_neuron in range(number_neurons):
107
                obervation_column.append(heaviside(self.output[iteration][index_neuron]));
108
            
109
            observation_matrix.append(obervation_column);
110
        
111
        return observation_matrix;
112
    
113
    
114 View Code Duplication
    def __allocate_neuron_patterns(self, start_iteration, stop_iteration):
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated in your project.
Loading history...
115
        """!
116
        @brief Allocates observation transposed matrix of neurons that is limited by specified periods of simulation.
117
        @details Matrix where state of each neuron is denoted by zero/one in line with Heaviside function on each iteration.
118
        
119
        @return (list) Transposed observation matrix that is limited by specified periods of simulation.
120
        
121
        """
122
        
123
        pattern_matrix = [];
124
        for index_neuron in range(len(self.output[0])):
125
            pattern_neuron = [];
126
            for iteration in range(start_iteration, stop_iteration):
127
                pattern_neuron.append(heaviside(self.output[iteration][index_neuron]))
128
            
129
            pattern_matrix.append(pattern_neuron);
130
        
131
        return pattern_matrix;
132
    
133
    
134
    def allocate_sync_ensembles(self, steps):
135
        """!
136
        @brief Allocate clusters in line with ensembles of synchronous neurons where each synchronous ensemble corresponds to only one cluster.
137
               
138
        @param[in] steps (double): Amount of steps from the end that is used for analysis. During specified period chaotic neural network should have stable output
139
                    otherwise inccorect results are allocated.
140
        
141
        @return (list) Grours (lists) of indexes of synchronous oscillators.
142
                For example [ [index_osc1, index_osc3], [index_osc2], [index_osc4, index_osc5] ].
143
        
144
        """
145
        
146
        iterations = steps;
147
        if (iterations >= len(self.output)):
148
            iterations = len(self.output);
149
        
150
        ensembles = [];
151
152
        start_iteration = len(self.output) - iterations;
153
        end_iteration = len(self.output);
154
        
155
        pattern_matrix = self.__allocate_neuron_patterns(start_iteration, end_iteration);
156
        
157
        ensembles.append( [0] );
158
        
159
        for index_neuron in range(1, len(self.output[0])):
160
            neuron_pattern = pattern_matrix[index_neuron][:];
161
            
162
            neuron_assigned = False;
163
            
164
            for ensemble in ensembles:
165
                ensemble_pattern = pattern_matrix[ensemble[0]][:];
166
167
                if (neuron_pattern == ensemble_pattern):
168
                    ensemble.append(index_neuron);
169
                    neuron_assigned = True;
170
                    break;
171
            
172
            if (neuron_assigned is False):
173
                ensembles.append( [index_neuron] );
174
        
175
        return ensembles;
176
177
178
class cnn_visualizer:
179
    """!
180
    @brief Visualizer of output dynamic of chaotic neural network (CNN).
181
    
182
    """
183
    
184
    @staticmethod
185
    def show_output_dynamic(cnn_output_dynamic):
186
        """!
187
        @brief Shows output dynamic (output of each neuron) during simulation.
188
        
189
        @param[in] cnn_output_dynamic (cnn_dynamic): Output dynamic of the chaotic neural network.
190
        
191
        @see show_dynamic_matrix
192
        @see show_observation_matrix
193
        
194
        """
195
        
196
        draw_dynamics(cnn_output_dynamic.time, cnn_output_dynamic.output, x_title = "t", y_title = "x");
197
    
198
    
199
    @staticmethod
200
    def show_dynamic_matrix(cnn_output_dynamic):
201
        """!
202
        @brief Shows output dynamic as matrix in grey colors.
203
        @details This type of visualization is convenient for observing allocated clusters.
204
        
205
        @param[in] cnn_output_dynamic (cnn_dynamic): Output dynamic of the chaotic neural network.
206
        
207
        @see show_output_dynamic
208
        @see show_observation_matrix
209
        
210
        """
211
        
212
        network_dynamic = numpy.array(cnn_output_dynamic.output);
213
        
214
        plt.imshow(network_dynamic.T, cmap = plt.get_cmap('gray'), interpolation='None', vmin = 0.0, vmax = 1.0); 
215
        plt.show();
216
    
217
    
218
    @staticmethod
219
    def show_observation_matrix(cnn_output_dynamic):
220
        """!
221
        @brief Shows observation matrix as black/white blocks.
222
        @details This type of visualization is convenient for observing allocated clusters.
223
        
224
        @param[in] cnn_output_dynamic (cnn_dynamic): Output dynamic of the chaotic neural network.
225
        
226
        @see show_output_dynamic
227
        @see show_dynamic_matrix
228
        
229
        """
230
        
231
        observation_matrix = numpy.array(cnn_output_dynamic.allocate_observation_matrix());
232
        plt.imshow(observation_matrix.T, cmap = plt.get_cmap('gray'), interpolation='None', vmin = 0.0, vmax = 1.0); 
233
        plt.show();
234
235
236
class cnn_network:
237
    """!
238
    @brief Chaotic neural network based on system of logistic map where clustering phenomenon can be observed.
239
    
240
    Example:
241
    @code
242
        # load stimulus from file
243
        stimulus = read_sample(SIMPLE_SAMPLES.SAMPLE_SIMPLE1);
244
        
245
        # create chaotic neural network, amount of neurons should be equal to amout of stimulus
246
        network_instance = cnn_network(len(stimulus));
247
        
248
        # simulate it during 100 steps
249
        output_dynamic = network_instance.simulate(steps, stimulus);
250
        
251
        # display output dynamic of the network
252
        cnn_visualizer.show_output_dynamic(output_dynamic);
253
        
254
        # dysplay dynamic matrix and observation matrix to show clustering
255
        # phenomenon.
256
        cnn_visualizer.show_dynamic_matrix(output_dynamic);
257
        cnn_visualizer.show_observation_matrix(output_dynamic);
258
    @endcode
259
    
260
    """
261
    
262
    def __init__(self, num_osc, conn_type = type_conn.ALL_TO_ALL, amount_neighbors = 3):
263
        """!
264
        @brief Constructor of chaotic neural network.
265
        
266
        @param[in] num_osc (uint): Amount of neurons in the chaotic neural network.
267
        @param[in] conn_type (type_conn): CNN type connection for the network.
268
        @param[in] amount_neighbors (uint): k-nearest neighbors for calculation scaling constant of weights.
269
        
270
        """
271
        
272
        self.__num_osc = num_osc;
273
        self.__conn_type = conn_type;
274
        self.__amount_neighbors = amount_neighbors;
275
        
276
        self.__average_distance = 0.0;
277
        self.__weights = None;
278
        self.__weights_summary = None;
279
        
280
        self.__location = None;     # just for network visualization
281
        
282
        random.seed();
283
        self.__output = [ random.random() for _ in range(num_osc) ];
284
    
285
    
286
    def __len__(self):
287
        """!
288
        @brief Returns size of the chaotic neural network that is defined by amount of neurons.
289
        
290
        """
291
        return self.__num_osc;
292
    
293
    
294
    def simulate(self, steps, stimulus):
295
        """!
296
        @brief Simulates chaotic neural network with extrnal stimulus during specified steps.
297
        @details Stimulus are considered as a coordinates of neurons and in line with that weights
298
                 are initialized.
299
        
300
        @param[in] steps (uint): Amount of steps for simulation.
301
        @param[in] stimulus (list): Stimulus that are used for simulation.
302
        
303
        @return (cnn_dynamic) Output dynamic of the chaotic neural network.
304
        
305
        """
306
        
307
        self.__create_weights(stimulus);
308
        self.__location = stimulus;
309
        
310
        dynamic = cnn_dynamic([], []);
311
        dynamic.output.append(self.__output);
312
        dynamic.time.append(0);
313
        
314
        for step in range(1, steps, 1):
315
            self.__output = self.__calculate_states();
316
            
317
            dynamic.output.append(self.__output);
318
            dynamic.time.append(step);
319
            
320
        return dynamic;
321
    
322
    
323
    def __calculate_states(self):
324
        """!
325
        @brief Calculates new state of each neuron.
326
        @detail There is no any assignment.
327
        
328
        @return (list) Returns new states (output).
329
        
330
        """
331
        
332
        output = [ 0.0 for _ in range(self.__num_osc) ];
333
        
334
        for i in range(self.__num_osc):
335
            output[i] = self.__neuron_evolution(i);
336
        
337
        return output;
338
    
339
    
340
    def __neuron_evolution(self, index):
341
        """!
342
        @brief Calculates state of the neuron with specified index.
343
        
344
        @param[in] index (uint): Index of neuron in the network.
345
        
346
        @return (double) New output of the specified neuron.
347
        
348
        """
349
        value = 0.0;
350
        
351
        for index_neighbor in range(self.__num_osc):
352
            value += self.__weights[index][index_neighbor] * (1.0 - 2.0 * (self.__output[index_neighbor] ** 2));
353
        
354
        return value / self.__weights_summary[index];
355
    
356
    
357
    def __create_weights(self, stimulus):
358
        """!
359
        @brief Create weights between neurons in line with stimulus.
360
        
361
        @param[in] stimulus (list): External stimulus for the chaotic neural network.
362
        
363
        """
364
        
365
        self.__average_distance = average_neighbor_distance(stimulus, self.__amount_neighbors);
366
        
367
        self.__weights = [ [ 0.0 for _ in range(len(stimulus)) ] for _ in range(len(stimulus)) ];
368
        self.__weights_summary = [ 0.0 for _ in range(self.__num_osc) ];
369
        
370
        if (self.__conn_type == type_conn.ALL_TO_ALL):
371
            self.__create_weights_all_to_all(stimulus);
372
        
373
        elif (self.__conn_type == type_conn.TRIANGULATION_DELAUNAY):
374
            self.__create_weights_delaunay_triangulation(stimulus);
375
    
376
    
377
    def __create_weights_all_to_all(self, stimulus):
378
        """!
379
        @brief Create weight all-to-all structure between neurons in line with stimulus.
380
        
381
        @param[in] stimulus (list): External stimulus for the chaotic neural network.
382
        
383
        """
384
        
385
        for i in range(len(stimulus)):
386
            for j in range(i + 1, len(stimulus)):
387
                weight = self.__calculate_weight(stimulus[i], stimulus[j]);
388
                
389
                self.__weights[i][j] = weight;
390
                self.__weights[j][i] = weight;
391
                
392
                self.__weights_summary[i] += weight;
393
                self.__weights_summary[j] += weight;
394
    
395
    
396
    def __create_weights_delaunay_triangulation(self, stimulus):
397
        """!
398
        @brief Create weight Denlauny triangulation structure between neurons in line with stimulus.
399
        
400
        @param[in] stimulus (list): External stimulus for the chaotic neural network.
401
        
402
        """
403
        
404
        points = numpy.array(stimulus);
405
        triangulation = Delaunay(points);
406 View Code Duplication
        
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated in your project.
Loading history...
407
        for triangle in triangulation.simplices:
408
            for index_tri_point1 in range(len(triangle)):
409
                for index_tri_point2 in range(index_tri_point1 + 1, len(triangle)):
410
                    index_point1 = triangle[index_tri_point1];
411
                    index_point2 = triangle[index_tri_point2];
412
                    
413
                    weight = self.__calculate_weight(stimulus[index_point1], stimulus[index_point2]);
414
                    
415
                    self.__weights[index_point1][index_point2] = weight;
416
                    self.__weights[index_point2][index_point1] = weight;
417
                    
418
                    self.__weights_summary[index_point1] += weight;
419
                    self.__weights_summary[index_point2] += weight;
420
    
421
    
422
    def __calculate_weight(self, stimulus1, stimulus2):
423
        """!
424
        @brief Calculate weight between neurons that have external stimulus1 and stimulus2.
425
        
426
        @param[in] stimulus1 (list): External stimulus of the first neuron.
427
        @param[in] stimulus2 (list): External stimulus of the second neuron.
428
        
429
        @return (double) Weight between neurons that are under specified stimulus.
430
        
431
        """
432
        
433
        distance = euclidean_distance_sqrt(stimulus1, stimulus2);
434
        return math.exp(-distance / (2.0 * self.__average_distance));
435
436
    
437
    def show_network(self):
438
        """!
439
        @brief Shows structure of the network: neurons and connections between them.
440
        
441
        """
442
        
443
        dimension = len(self.__location[0]);
444
        if ( (dimension != 3) and (dimension != 2) ):
445
            raise NameError('Network that is located in different from 2-d and 3-d dimensions can not be represented');
446
447
        (fig, axes) = self.__create_surface(dimension);
0 ignored issues
show
Unused Code introduced by
The variable fig seems to be unused.
Loading history...
448
        
449
        for i in range(0, self.__num_osc, 1):
450
            if (dimension == 2):
451
                axes.plot(self.__location[i][0], self.__location[i][1], 'bo');  
452
                for j in range(i, self.__num_osc, 1):    # draw connection between two points only one time
453
                    if (self.__weights[i][j] > 0.0):
454
                        axes.plot([self.__location[i][0], self.__location[j][0]], [self.__location[i][1], self.__location[j][1]], 'b-', linewidth = 0.5);
455
            
456
            elif (dimension == 3):
457
                axes.scatter(self.__location[i][0], self.__location[i][1], self.__location[i][2], c = 'b', marker = 'o');
458
                
459
                for j in range(i, self._num_osc, 1):    # draw connection between two points only one time
0 ignored issues
show
Bug introduced by
The Instance of cnn_network does not seem to have a member named _num_osc.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
460
                    if (self.__weights[i][j] > 0.0):
461
                        axes.plot([self.__location[i][0], self.__location[j][0]], [self.__location[i][1], self.__location[j][1]], [self.__location[i][2], self.__location[j][2]], 'b-', linewidth = 0.5);
462
                
463
        plt.grid();
464
        plt.show();
465
    
466
    
467
    def __create_surface(self, dimension):
468
        """!
469
        @brief Prepares surface for showing network structure in line with specified dimension.
470
        
471
        @param[in] dimension (uint): Dimension of processed data (external stimulus).
472
        
473
        @return (tuple) Description of surface for drawing network structure.
474
        
475
        """
476
        
477
        rcParams['font.sans-serif'] = ['Arial'];
478
        rcParams['font.size'] = 12;
479
480
        fig = plt.figure();
481
        axes = None;
482
        if (dimension == 2):
483
            axes = fig.add_subplot(111);
484
        elif (dimension == 3):
485
            axes = fig.gca(projection='3d');
486
        
487
        surface_font = FontProperties();
488
        surface_font.set_name('Arial');
489
        surface_font.set_size('12');
490
        
491
        return (fig, axes);