Completed
Pull Request — master (#305)
by Bart
01:50
created

fuel.utils.open_()   B

Complexity

Conditions 6

Size

Total Lines 36

Duplication

Lines 0
Ratio 0 %
Metric Value
cc 6
dl 0
loc 36
rs 7.5384
1
"""Low-level utilities for reading a variety of source formats."""
0 ignored issues
show
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.text).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.svhn).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.cifar10).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.hdf5).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.mnist).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.caltech101_silhouettes).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.cifar100).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.billion).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.adult).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.binarized_mnist).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.iris).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.celeba).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
Bug introduced by
There seems to be a cyclic import (fuel.datasets -> fuel.datasets.dogs_vs_cats).

Cyclic imports may cause partly loaded modules to be returned. This might lead to unexpected runtime behavior which is hard to debug.

Loading history...
2
import codecs
3
import gzip
4
import io
5
import tarfile
6
import six
7
8
9
def open_(filename, mode='r', encoding='utf-8'):
10
    """Open a text file with UTF-8 and optional gzip compression.
11
12
    This function is useful when dealing with text files.
13
14
    Parameters
15
    ----------
16
    filename : str
17
        The filename to read.
18
    mode : str, optional
19
        The mode with which to open the file. Defaults to `r`.
20
    encoding : str, optional
21
        The encoding to use (see the codecs documentation_ for supported
22
        values). Defaults to `utf-8`.
23
24
    .. _documentation:
25
    https://docs.python.org/3/library/codecs.html#standard-encodings
26
27
    """
28
    if filename.endswith('.gz'):
29
        if six.PY2:
30
            zf = io.BufferedReader(gzip.open(filename, mode))
31
            if encoding:
32
                return codecs.getreader(encoding)(zf)
33
            else:
34
                return zf
35
        else:
36
            return io.BufferedReader(gzip.open(filename, mode,
37
                                               encoding=encoding))
38
    if six.PY2:
39
        if encoding:
40
            return codecs.open(filename, mode, encoding=encoding)
41
        else:
42
            return open(filename, mode)
43
    else:
44
        return open(filename, mode, encoding=encoding)
45
46
47
def tar_open(f):
48
    """Open either a filename or a file-like object as a TarFile.
49
50
    Parameters
51
    ----------
52
    f : str or file-like object
53
        The filename or file-like object from which to read.
54
55
    Returns
56
    -------
57
    TarFile
58
        A `TarFile` instance.
59
60
    """
61
    if isinstance(f, six.string_types):
62
        return tarfile.open(name=f)
63
    else:
64
        return tarfile.open(fileobj=f)
65