Completed
Pull Request — master (#305)
by Bart
01:27
created

fuel.utils.open_()   B

Complexity

Conditions 6

Size

Total Lines 37

Duplication

Lines 0
Ratio 0 %
Metric Value
cc 6
dl 0
loc 37
rs 7.5384
1
"""Low-level utilities for reading a variety of source formats."""
0 ignored issues
show
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.text).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.svhn).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.cifar10).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.hdf5).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.mnist).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.caltech101_silhouettes).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.cifar100).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.billion).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.adult).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.binarized_mnist).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.iris).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.celeba).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.dogs_vs_cats).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
2
import codecs
3
import gzip
4
import io
5
import tarfile
6
import six
7
8
9
def open_(filename, mode='r', encoding=None):
10
    """Open a text file with encoding and optional gzip compression.
11
12
    Note that on legacy Python any encoding other than ``None`` or opening
13
    GZipped files will return an unpicklable file-like object.
14
15
    Parameters
16
    ----------
17
    filename : str
18
        The filename to read.
19
    mode : str, optional
20
        The mode with which to open the file. Defaults to `r`.
21
    encoding : str, optional
22
        The encoding to use (see the codecs documentation_ for supported
23
        values). Defaults to ``None``.
24
25
    .. _documentation:
26
    https://docs.python.org/3/library/codecs.html#standard-encodings
27
28
    """
29
    if filename.endswith('.gz'):
30
        if six.PY2:
31
            zf = io.BufferedReader(gzip.open(filename, mode))
32
            if encoding:
33
                return codecs.getreader(encoding)(zf)
34
            else:
35
                return zf
36
        else:
37
            return io.BufferedReader(gzip.open(filename, mode,
38
                                               encoding=encoding))
39
    if six.PY2:
40
        if encoding:
41
            return codecs.open(filename, mode, encoding=encoding)
42
        else:
43
            return open(filename, mode)
44
    else:
45
        return open(filename, mode, encoding=encoding)
46
47
48
def tar_open(f):
49
    """Open either a filename or a file-like object as a TarFile.
50
51
    Parameters
52
    ----------
53
    f : str or file-like object
54
        The filename or file-like object from which to read.
55
56
    Returns
57
    -------
58
    TarFile
59
        A `TarFile` instance.
60
61
    """
62
    if isinstance(f, six.string_types):
63
        return tarfile.open(name=f)
64
    else:
65
        return tarfile.open(fileobj=f)
66