Completed
Push — master ( 980041...30b693 )
by
unknown
10s
created

BaseOutputTest.test_save_file()   A

Complexity

Conditions 1

Size

Total Lines 4

Duplication

Lines 0
Ratio 0 %
Metric Value
dl 0
loc 4
rs 10
cc 1
1
import os
0 ignored issues
show
Coding Style introduced by
This module should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
2
import unittest
3
4
import elasticsearch
5
6
from topik.fileio.base_output import load_output
7
from topik.fileio.reader import read_input
8
from topik.fileio.tests import test_data_path
9
from topik.fileio.out_elastic import ElasticSearchOutput
10
from topik.fileio.out_memory import InMemoryOutput
11
12
INDEX = "topik_unittest"
13
SAVE_FILENAME = "test_save.topikdata"
14
CONTENT_FIELD = "abstract"
15
16
# make logging quiet during testing, to keep Travis CI logs short.
17
import logging
18
logging.basicConfig()
19
logging.getLogger('elasticsearch').setLevel(logging.ERROR)
20
logging.getLogger('urllib3').setLevel(logging.ERROR)
21
22
23
class BaseOutputTest(object):
0 ignored issues
show
Coding Style introduced by
This class should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
24
    test_raw_data = None
25
26
    def test_get_filtered_data(self):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
27
        data = list(self.test_raw_data.get_filtered_data(CONTENT_FIELD))
28
        self.assertEqual(len(data), 100)
0 ignored issues
show
Bug introduced by
The Instance of BaseOutputTest does not seem to have a member named assertEqual.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
29
        self.assertFalse(data[0] == data[1])
0 ignored issues
show
Bug introduced by
The Instance of BaseOutputTest does not seem to have a member named assertFalse.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
30
31
    def test_save_file(self):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
32
        self.test_raw_data.save(SAVE_FILENAME)
33
        self.assertTrue(os.path.exists(SAVE_FILENAME))
0 ignored issues
show
Bug introduced by
The Instance of BaseOutputTest does not seem to have a member named assertTrue.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
34
        os.remove(SAVE_FILENAME)
35
36
    def test_load_file(self):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
37
        self.test_raw_data.save(SAVE_FILENAME)
38
        self.test_raw_data = load_output(SAVE_FILENAME)
39
        data = list(self.test_raw_data.get_filtered_data(CONTENT_FIELD))
40
        self.assertEqual(len(data), 100)
0 ignored issues
show
Bug introduced by
The Instance of BaseOutputTest does not seem to have a member named assertEqual.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
41
        os.remove(SAVE_FILENAME)
42
43
    def test_get_date_filtered_data(self):
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
44
        result_list = list(self.test_raw_data.get_date_filtered_data(field_to_get=CONTENT_FIELD,
45
                                                                     start=1975,
46
                                                                     end=1999,
47
                                                                     filter_field="year"))
48
        self.assertEqual(len(result_list), 25)
0 ignored issues
show
Bug introduced by
The Instance of BaseOutputTest does not seem to have a member named assertEqual.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
49
        self.assertTrue(-1611117933394825767 in [int(item[0]) for item in
0 ignored issues
show
Bug introduced by
The Instance of BaseOutputTest does not seem to have a member named assertTrue.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
50
                        result_list])
51
52
class TestInMemoryOutput(unittest.TestCase, BaseOutputTest):
0 ignored issues
show
Coding Style introduced by
This class should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
53
    def setUp(self):
54
        self.test_raw_data = InMemoryOutput()
55
        self.test_raw_data.import_from_iterable(read_input(
56
            '{}/test_data_json_stream.json'.format(test_data_path)),
57
            field_to_hash=CONTENT_FIELD)
58
59
class TestElasticSearchOutput(unittest.TestCase, BaseOutputTest):
0 ignored issues
show
Coding Style introduced by
This class should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
60
    def setUp(self):
61
        self.test_raw_data = ElasticSearchOutput(
62
            source='localhost',
63
            index=INDEX,
64
            content_field='abstract'
65
        )
66
        self.test_raw_data.import_from_iterable(read_input(
67
            '{}/test_data_json_stream.json'.format(test_data_path)),
68
            field_to_hash=CONTENT_FIELD)
69
70
    def tearDown(self):
71
        instance = elasticsearch.Elasticsearch("localhost")
72
        instance.indices.delete(INDEX)
73
        if instance.indices.exists("{}_year_alias_date".format(INDEX)):
74
            instance.indices.delete("{}_year_alias_date".format(INDEX))
75
76