Passed
Push — develop ( be29f2...b2a598 )
by Dylan
05:09 queued 02:15
created

js/crawler.js   F

Complexity

Total Complexity 117
Complexity/F 2.79

Size

Lines of Code 564
Function Count 42

Duplication

Duplicated Lines 0
Ratio 0 %

Importance

Changes 3
Bugs 0 Features 0
Metric Value
wmc 117
dl 0
loc 564
rs 3.12
c 3
b 0
f 0
cc 0
nc 1
mnd 4
bc 48
fnc 42
bpm 1.1428
cpm 2.7857
noi 59

32 Functions

Rating   Name   Duplication   Size   Complexity  
A crawler_painter.update_header 0 7 4
B crawler.is_external 0 12 7
A crawler_painter.reset_table 0 10 2
C crawler_painter.add_row 0 22 7
A crawler_painter.create 0 49 2
A crawler.ignore_url 0 7 3
A crawler.get_word_count 0 7 3
A crawler.trigger 0 4 3
A crawler.fetch_links 0 11 1
A crawler.is_file 0 4 1
A crawler.add_ignore_path 0 4 1
A crawler_painter.create_link 0 7 3
A crawler.set_crawl_id 0 4 1
A crawler.sanitize 0 12 4
A crawler.get_proxy 0 3 1
B crawler.regiser_test 0 7 7
A crawler.can_crawl 0 5 2
A crawler.get_test_by_name 0 4 3
A crawler.is_anchor 0 3 1
A crawler.run_tests 0 7 2
A crawler.get_domain 0 5 3
B crawler_painter.set_type 0 11 6
A crawler.add_row 0 3 1
A crawler_painter.init 0 4 2
A crawler.que_url 0 5 4
A crawler_painter.get_container_by_name 0 3 3
B crawler.fetch_and_test 0 35 5
B crawler_painter.create_status 0 26 5
A crawler.strip_img_src 0 3 1
A crawler.on 0 4 2
A crawler.init 0 20 4
A crawler.set_ignore_paths 0 4 1

How to fix   Complexity   

Complexity

Complex classes like js/crawler.js often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

1
const crawler = {
2
3
    que             : [],
4
    tested          : [],
5
    crawling        : [],
6
    tests           : [],
7
    ignore_paths    : [],
8
    crawl_id        : undefined,
9
    events          : {},
10
    linked_from     : {},
11
    useragent       : 'desktop',
12
13
    /**
14
     * Register a test to run.
15
     *
16
     * @param {string} name
17
     * @param {string} title
18
     * @param {Array} headers
19
     * @param {string} callable
20
     * @returns {boolean}
21
     * @throws Exception
22
     */
23
    regiser_test: function(name, title, headers, callable){
24
        if(name == undefined || this.get_test_by_name(name)) throw 'Invalid name specified for your test';
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
25
        if(title == undefined) throw 'Title not specified';
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
26
        if(!(headers instanceof Array) || headers.length < 1) throw 'Headers array is invalid';
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
27
        if(typeof callable != 'function') return crawler_painter.create(name, title, headers);
2 ignored issues
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
Bug introduced by
The local (let) variable crawler_painter is used before it is defined. This will cause a reference error.
Loading history...
28
        this.tests.push({name: name, title: title, callback: callable, cont:crawler_painter.create(name, title, headers)});
0 ignored issues
show
Best Practice introduced by
There is no return statement in this branch, but you do return something in other branches. Did you maybe miss it? If you do not want to return anything, consider adding return undefined; explicitly.
Loading history...
29
    },
30
31
    /**
32
     * Return a registered test by name
33
     *
34
     * @param {string} name
35
     * @returns {object|false}
36
     */
37
    get_test_by_name: function(name){
38
        for(var t in this.test) if(this.tests[t]['name'] == name) return this.tests[t];
2 ignored issues
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
39
        return false;
40
    },
41
42
    /**
43
     * Check if the url passed is valid for crawling, if so and it hasn't
44
     * been added or crawled before, add it to the que
45
     *
46
     * Returns false if failed to add to que
47
     *
48
     * @param {string} url
49
     * @returns {boolean|undefined}
50
     */
51
    que_url: function(url){
52
        var sanitized = this.sanitize(url);
53
        if( !this.can_crawl(url) || this.que.indexOf(sanitized) > -1 || !this.can_crawl(sanitized)) return false;
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
54
        this.que.push(sanitized);
0 ignored issues
show
Best Practice introduced by
There is no return statement in this branch, but you do return something in other branches. Did you maybe miss it? If you do not want to return anything, consider adding return undefined; explicitly.
Loading history...
55
    },
56
57
    /**
58
     *  Clean up a url so it becomes relative and standardized
59
     *
60
     * @param {string} url
61
     * @returns {string}
62
     */
63
    sanitize: function(url){
64
        if(url == undefined) return '';
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
65
66
        url = url
67
            .replace(/https?:\/\/[^\/]+/i, '')
68
            .replace(/^\/|\/$/g, '').split('#')[0];
69
70
        if( url.slice(-1) == '?' ) url = url.slice(0, -1);
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
71
        if( url.length < 1 ) url = '/';
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
72
73
        return url;
74
    },
75
76
    /**
77
     * Get the domain for the passed url
78
     *
79
     * @param {string} url
80
     * @returns {string}
81
     */
82
    get_domain: function(url){
83
        if( !url ) return '';
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
84
        if( url.indexOf("://") > -1 ) return url.split('/')[2].split(':')[0];
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
85
        else return url.split('/')[0].split(':')[0];
86
    },
87
88
    /**
89
     * Checks if the passed url should be ignored or not
90
     *
91
     * @param {string} url
92
     * @returns {boolean}
93
     */
94
    ignore_url: function( url ){
95
        for(var regex in this.ignore_paths) {
1 ignored issue
show
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
96
            var reg = new RegExp(this.ignore_paths[regex], 'i');
97
            if( url.match(reg) != null ) return true;
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
98
        }
99
        return false;
100
    },
101
102
    /**
103
     * Add a path to ignore when crawler
104
     * Note: Paths can be in regex format
105
     *
106
     * @param {string} path
107
     * @returns {crawler}
108
     */
109
    add_ignore_path: function(path){
110
        this.ignore_paths.push(path);
111
        return this;
112
    },
113
114
    /**
115
     * Update all ignore paths to the paths specified
116
     * Note: Path can be in regex format
117
     *
118
     * @param paths
119
     * @returns {crawler}
120
     */
121
    set_ignore_paths: function(paths){
122
        this.ignore_paths = paths;
123
        return this;
124
    },
125
126
    /**
127
     * Sets the crawl id
128
     *
129
     * @param crawl_id
130
     * @returns {crawler}
131
     */
132
    set_crawl_id: function(crawl_id){
133
        this.crawl_id = crawl_id;
134
        return this;
135
    },
136
137
    /**
138
     * Does some soft checks to determine if url is a valid candidate for crawling
139
     *
140
     * @param {string} url
141
     * @returns {boolean}
142
     */
143
    can_crawl: function(url){
144
        if(url == undefined) return false;
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
145
        return !(this.crawling.indexOf(url) >= 0 || this.tested.indexOf(url) >= 0 ||
146
                    this.is_file(url) || this.ignore_url(url) || this.is_external(url));
147
    },
148
149
    /**
150
     * Does a soft check for the url passed and checks if it's a file
151
     * by checking if it has an extension and if the extension contains 'html'
152
     *
153
     * @param {string} url
154
     * @returns {boolean}
155
     */
156
    is_file: function(url){
157
        var split = this.sanitize( url ).split( '.' );
158
        return split.length > 1 && split.pop().indexOf( 'html' ) < 0;
159
    },
160
161
    /**
162
     * Does some soft checking for the url passed to see if it's external
163
     * Note: If the url is internal but redirects to an external source, we wown't detect it here
164
     *
165
     * @param {string} url
166
     * @returns {boolean}
167
     */
168
    is_external: function(url){
169
        // Starts with / or # or doesn't have :// in it has to be internal
170
        if( url.length < 1 || url[0] == '/' || url[0] == '#' || url.indexOf('://') < 0 ) return false;
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
171
172
        // If we removed the domain and the url is still the same then it's an internal link without the leading /
173
        if( url == this.sanitize( url ) ) return false;
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
174
175
        // The domain is the same the domain we're running this script on
176
        if( this.get_domain( url ) == location.hostname ) return false;
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
177
178
        return true;
179
    },
180
181
    /**
182
     * Checks if the href passed is an anchor link for url passed.
183
     *
184
     * @param {string} href
185
     * @param {string} url
186
     * @return {boolean}
187
     */
188
    is_anchor: function(href, url){
189
        return href.indexOf('#') >= 0 && this.sanitize(href) == this.sanitize(url);
190
    },
191
192
    /**
193
     * Fetch the next url from the que and run the tests on it
194
     */
195
    fetch_and_test: function(){
196
        if( !this.que || this.que.length < 1 || this.que.length < 1 || $.active > 2 ) return false;
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
197
198
        var url = this.que.pop();
199
        this.crawling.push(url);
200
201
        $.ajax({
202
            url: this.get_proxy( url ), data: { agent: this.useragent }, accepts: 'json', dataType: 'json'
203
        })
204
            .done(function( result ) {
205
                if(result['headers'] && result['body'] && result['body'].toLowerCase().indexOf('<head') >= 0) {
206
                    if( !crawler.is_external(result['url_fetched']) ) {
207
                        url = crawler.sanitize(result['url_fetched']);
208
                        if(crawler.tested.indexOf(url) >= 0){
209
                            this.skipped = true;
210
                            return true;
211
                        }
212
213
                        var html = $(crawler.strip_img_src(result['body']));
214
                        crawler.trigger('CRAWL_BEFORE_TESTS', [url]);
215
                        crawler.fetch_links(html, url);
216
                        crawler.run_tests(url, html, result['headers'], result['field_data'], result['phrases']);
217
                        crawler.trigger('CRAWL_AFTER_TESTS', [url]);
218
                        return true;
219
                    }
220
                }
221
                crawler.trigger('CRAWL_LOAD_FAILED', [url]);
0 ignored issues
show
Best Practice introduced by
There is no return statement in this branch, but you do return something in other branches. Did you maybe miss it? If you do not want to return anything, consider adding return undefined; explicitly.
Loading history...
222
            })
223
            .fail( function(){ crawler.trigger('CRAWL_LOAD_FAILED', [url]); })
224
            .always( function(){
225
                crawler.trigger('CRAWL_FINISHED', [url]);
226
                if((this.hasOwnProperty('skipped') && this.skipped) || crawler.tested.indexOf(url) < 0 )
227
                    crawler.tested.push(url)
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
228
            });
0 ignored issues
show
Best Practice introduced by
There is no return statement in this branch, but you do return something in other branches. Did you maybe miss it? If you do not want to return anything, consider adding return undefined; explicitly.
Loading history...
229
    },
230
231
    /**
232
     * Check for links in the html of the rendered page so we add them to the que
233
     * and also map how pages are linked to each other
234
     *
235
     * @param {jQuery} html
236
     * @param {string} url
237
     */
238
    fetch_links: function(html, url){
239
        $.each(html.find('a'), function(){
240
            var href    = $(this).attr('href'),
241
                link    = crawler.sanitize(href);
242
243
            crawler.que_url( href );
244
245
            if(!crawler.linked_from.hasOwnProperty(link)) crawler.linked_from[link] = [url];
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
246
            else if( crawler.linked_from[link].indexOf(url) < 0 ) crawler.linked_from[link].push(url);
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
247
        });
248
    },
249
250
    /**
251
     * Run the registered tests
252
     *
253
     * @param {string} url
254
     * @param {jQuery} html
255
     * @param {Array} headers
256
     * @param {Array} field_data
257
     * @param {Array} phrases
258
     */
259
    run_tests: function(url, html, headers, field_data, phrases){
260
        for(var t in this.tests) {
1 ignored issue
show
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
261
            this.trigger('before'+this.tests[t]['name'], [url, html, headers, field_data, phrases]);
262
            this.tests[t]['callback'].apply(this.tests[t], [this.tests[t]['cont'], url, html, headers, field_data, phrases]);
263
            this.trigger('after'+this.tests[t]['name'], [url, html, headers, field_data, phrases]);
264
        }
265
    },
266
267
    /**
268
     * Trigger event callback and pass on the data
269
     *
270
     * @param {string} event
271
     * @param {*} data
272
     */
273
    trigger: function(event, data){
274
        if(this.events.hasOwnProperty(event))
275
            for(var e in this.events[event]) this.events[event][e].apply(this, data);
2 ignored issues
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
276
    },
277
278
    /**
279
     * Register callback on action
280
     *
281
     * @param {string} event
282
     * @param {function} callback
283
     * @returns {crawler}
284
     */
285
    on: function(event, callback){
286
        if(!this.events.hasOwnProperty(event)) this.events[event] = [];
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
287
        this.events[event].push(callback);
288
    },
289
290
    /**
291
     * Strip out src=<anything> so that we avoid loading the images
292
     * on the pages
293
     *
294
     * @param {string}html
295
     * @returns {string}
296
     */
297
    strip_img_src: function(html){
298
        return html.replace( /(src).*?=(['|"].*?['|"])/ig, '' );
299
    },
300
301
    /**
302
     * Return the proxy url to test the passed url
303
     *
304
     * @param {$string} url
305
     * @returns {string}
306
     */
307
    get_proxy: function(url){
308
        return location.protocol + '//' + location.hostname + '/seotest/getPageData?u='+url;
309
    },
310
311
    /**
312
     * @see crawler_painter.add_row(name, data)
313
     * @param {string} name
314
     * @param {Array} data
315
     */
316
    add_row: function(name, data){
317
        crawler_painter.add_row(name, data);
1 ignored issue
show
Bug introduced by
The local (let) variable crawler_painter is used before it is defined. This will cause a reference error.
Loading history...
318
    },
319
320
    /**
321
     * Returns the word count for a given set of sentences or string
322
     *
323
     * @param {string|array} data
324
     * @returns {number}
325
     */
326
    get_word_count: function(data){
327
        if( typeof data === 'string' ) return data.split(' ').length;
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
328
329
        var count = 0;
330
        for( var d in data ) count += data[d].split(' ').length;
2 ignored issues
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
331
        return count;
332
    },
333
334
    /**
335
     * Start the crawler
336
     *
337
     * @param {object} settings
338
     * @throws Exception
339
     */
340
    init: function(settings){
341
        this.trigger('BEFORE_INIT', []);
342
343
        if(settings.hasOwnProperty('crawl_id')) this.set_crawl_id(settings['crawl_id']);
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
344
        if(settings.hasOwnProperty('ignore_paths')) this.set_ignore_paths(settings['ignore_paths']);
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
345
346
        if( !this.crawl_id ) throw "crawl_id must be specified";
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
347
348
        // When a crawl finishes, start a new one if there are any more urls to go through else stop the auto-restart
349
        this.on('CRAWL_FINISHED', function(){
350
            if( crawler.que.length > 0 ) crawler.fetch_and_test();
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
351
            else window.clearInterval(crawler.interval);
352
        });
353
354
        // Every second try to initialize a new crawl request just in-case something crashes
355
        this.interval = setInterval(function(){ crawler.fetch_and_test(); }, 1000);
356
357
        crawler_painter.init();
1 ignored issue
show
Bug introduced by
The local (let) variable crawler_painter is used before it is defined. This will cause a reference error.
Loading history...
358
        this.trigger('AFTER_INIT', []);
359
    }
360
};
361
362
const crawler_painter = {
363
364
    containers: [],
365
366
    /**
367
     * Create a result table for the provided tests
368
     *
369
     * @param {string} name
370
     * @param {string} title
371
     * @param {Array} headers
372
     * @return {jQuery}
373
     */
374
    create: function(name, title, headers){
375
        var container       = $('<div class="infobox" id="'+name+'"></div>'),
376
            header          = $('<div class="header clearfix"></div>'),
377
            count           = $('<div class="count left"></div>'),
378
            toggle_button   = $('<div class="icon toggle right closed"></div>').hide(),
379
            export_button   = $('<div class="icon export right"></div>').hide(),
380
            title_bar       = $('<h2 class="left">'+title+'</h2>'),
381
            table_cont      = $('<div class="tableCont" style="display:none;"></div>'),
382
            thead           = $('<tr></tr>'),
383
            table           = $('<table class="table table-hover table-condensed"></table>').append(['<thead></thead>', '<tbody></tbody>']);
384
385
        for(var h in headers) thead.append('<th>'+headers[h]+'</th>');
2 ignored issues
show
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
386
        table.find('thead').append(thead);
387
388
        header.append([count, title_bar, toggle_button, export_button]);
389
        table_cont.append(table);
390
        container.append([header, table_cont]);
391
392
        toggle_button.click(function(){
393
            crawler.trigger('TOGGLED', [name]);
394
            var $this   = $(this),
395
                css     = ($this.hasClass('closed')) ? 'opened' : 'closed';
396
            $this.parents('.infobox').find('.tableCont').slideToggle();
397
            $this.removeClass('opened closed');
398
            $this.addClass(css);
399
        });
400
401
        export_button.click(function(){
402
            crawler.trigger('BEFORE_EXPORT', [name]);
403
            var $this       = $(this),
404
                rows        = $this.parents( '.infobox' ).first().find( 'table tr' ),
405
                csvContent  = "data:text/csv;charset=utf-8,";
406
407
            $.each( rows, function(){
408
                var item = [];
409
                $.each( $(this).find( 'th, td' ), function(){ item.push( $(this).text() ); });
410
                csvContent += item.join(',') + "\n";
411
            });
412
413
            var link = document.createElement( 'a' );
414
            link.setAttribute( 'href', encodeURI( csvContent ) );
415
            link.setAttribute( 'download', name + '.csv' );
416
            link.click();
417
            crawler.trigger('AFTER_EXPORT', [name]);
418
        });
419
420
        this.containers.push({'name': name, 'container': container});
421
        return container;
422
    },
423
424
    /**
425
     * Add a row of data to the container which matches
426
     * the name provided
427
     *
428
     * @param {string} name
429
     * @param {Array} data
430
     */
431
    add_row: function(name, data){
432
        var cont    = this.get_container_by_name(name),
433
            table   = cont.find('tbody'),
434
            row     = $('<tr></tr>').appendTo(table),
435
            len     = table.find('tr').length;
436
437
        for(var d in data) row.append($('<td/>').append(data[d]));
2 ignored issues
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
438
439
        // Show icons if we have items
440
        if( len > 0 ){
441
            cont.find('.icon.export').fadeIn();
442
            cont.find('.icon.toggle').fadeIn();
443
        }
444
445
        cont.find('.count').html(len);
446
447
        // Set the header colour
448
        if( cont.find('td div.alert-danger').length > 0 ) crawler_painter.set_type(name, 'error');
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
449
        else if( cont.find('td div.alert-warning').length > 0 ) crawler_painter.set_type(name, 'warning');
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
450
        else if( cont.find('td div.alert-info').length > 0 ) crawler_painter.set_type(name, 'info');
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
451
        else if( cont.find('td div.alert-success').length > 0 ) crawler_painter.set_type(name, 'success');
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
452
    },
453
454
    /**
455
     * Reset the data inside of the table for the container named {name}
456
     *
457
     * @param {string} name
458
     * @param {string|undefined} type
459
     */
460
    reset_table: function(name, type){
461
        var cont = this.get_container_by_name(name);
462
463
        cont.find('tbody tr').remove();
464
        cont.find('.count').html('');
465
        cont.find('.icon.export').hide();
466
        cont.find('.icon.toggle').hide();
467
468
        if( type != undefined ) this.set_type(name, type);
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
469
    },
470
471
    /**
472
     * Create a status field to be used in the report rows
473
     *
474
     * @param string type
0 ignored issues
show
Documentation introduced by
The parameter string does not exist. Did you maybe forget to remove this comment?
Loading history...
475
     * @param string text
0 ignored issues
show
Documentation introduced by
The parameter string has already been documented on line 474. The second definition is ignored.
Loading history...
476
     */
477
    create_status: function(type, text){
478
        var ret = $('<div class="status-text alert"></div>');
479
        switch(type){
0 ignored issues
show
Coding Style introduced by
As per coding-style, switch statements should have a default case.
Loading history...
480
            case 'info':
481
                ret.addClass('alert-info');
482
                ret.append('<i class="glyphicon glyphicon-info-sign">&nbsp;</i>');
483
                break;
484
485
            case 'error':
486
                ret.addClass('alert-danger');
487
                ret.append('<i class="glyphicon glyphicon-exclamation-sign">&nbsp;</i>');
488
                break;
489
490
            case 'success':
491
                ret.addClass('alert-success');
492
                ret.append('<i class="glyphicon glyphicon-ok-sign">&nbsp;</i>');
493
                break;
494
495
            case 'warning':
496
                ret.addClass('alert-warning');
497
                ret.append('<i class="glyphicon glyphicon-warning-sign">&nbsp;</i>');
498
                break;
499
        }
500
        ret.append(text);
501
        return ret;
502
    },
503
504
    /**
505
     * Return the container matching the provided name
506
     *
507
     * @param {string} name
508
     * @returns {jQuery}
509
     */
510
    get_container_by_name: function(name){
511
        for(var c in this.containers) if(this.containers[c]['name'] == name) return this.containers[c]['container'];
2 ignored issues
show
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
Best Practice introduced by
There is no return statement in this branch, but you do return something in other branches. Did you maybe miss it? If you do not want to return anything, consider adding return undefined; explicitly.
Loading history...
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
512
    },
513
514
    /**
515
     * Set the type of the test so it's colour changes according
516
     *
517
     * @param {string} name
518
     * @param {string} type
519
     */
520
    set_type: function(name, type){
521
        var cont = this.get_container_by_name(name);
522
        cont.removeClass('blue red green yellow purple');
523
        switch(type){
524
            case 'info': return cont.addClass('blue');
525
            case 'error': return cont.addClass('red');
526
            case 'success': return cont.addClass('green');
527
            case 'warning': return cont.addClass('yellow');
528
            default: return cont.addClass('purple');
529
        }
530
    },
531
532
    /**
533
     * Update the header stats
534
     */
535
    update_header: function(){
536
        $('#leftcount').html(crawler.que.length);
537
        $('#donecount').html(crawler.tested.length);
538
539
        if(crawler.que.length > 0 ) $('#analyzestatus').html('Analyzing');
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
540
        else if(crawler.que.length < 1 && crawler.tested.length > 0) $('#analyzestatus').html('Finished');
1 ignored issue
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
541
    },
542
543
    /**
544
     * Returns a link out of the passed url
545
     *
546
     * @param {string} url
547
     * @returns {string}
548
     */
549
    create_link: function(url, anchor){
550
        anchor = (anchor) ? anchor : url;
551
        return '<a class="btn btn-link" href="'+url+'" title="'+anchor+'" target="_blank" rel="nofollow">'
552
                +'<span class="glyphicon glyphicon-new-window">&nbsp;</span>'+
553
                ((anchor.length > 29) ? anchor.substr(0, 27) + '...' : anchor)
554
                +'</a>';
555
    },
556
557
    /**
558
     * Initialize the painter
559
     */
560
    init:function(){
561
        for(var c in this.containers) $('#results_container').append(this.containers[c]['container']);
2 ignored issues
show
Coding Style Best Practice introduced by
Curly braces around statements make for more readable code and help prevent bugs when you add further statements.

Consider adding curly braces around all statements when they are executed conditionally. This is optional if there is only one statement, but leaving them out can lead to unexpected behaviour if another statement is added later.

Consider:

if (a > 0)
    b = 42;

If you or someone else later decides to put another statement in, only the first statement will be executed.

if (a > 0)
    console.log("a > 0");
    b = 42;

In this case the statement b = 42 will always be executed, while the logging statement will be executed conditionally.

if (a > 0) {
    console.log("a > 0");
    b = 42;
}

ensures that the proper code will be executed conditionally no matter how many statements are added or removed.

Loading history...
Complexity introduced by
A for in loop automatically includes the property of any prototype object, consider checking the key using hasOwnProperty.

When iterating over the keys of an object, this includes not only the keys of the object, but also keys contained in the prototype of that object. It is generally a best practice to check for these keys specifically:

var someObject;
for (var key in someObject) {
    if ( ! someObject.hasOwnProperty(key)) {
        continue; // Skip keys from the prototype.
    }

    doSomethingWith(key);
}
Loading history...
562
        crawler.on('CRAWL_FINISHED', this.update_header);
563
    }
564
};
565