Passed
Push — develop ( 610d34...d32a89 )
by Jace
01:29 queued 10s
created

doorstop.core.document.Document.new()   C

Complexity

Conditions 11

Size

Total Lines 54
Code Lines 21

Duplication

Lines 0
Ratio 0 %

Importance

Changes 0
Metric Value
eloc 21
dl 0
loc 54
rs 5.4
c 0
b 0
f 0
cc 11
nop 8

How to fix   Long Method    Complexity    Many Parameters   

Long Method

Small methods make your code easier to understand, in particular if combined with a good name. Besides, if your method is small, finding a good name is usually much easier.

For example, if you find yourself adding comments to a method's body, this is usually a good sign to extract the commented part to a new method, and use the comment as a starting point when coming up with a good name for this new method.

Commonly applied refactorings include:

Complexity

Complex classes like doorstop.core.document.Document.new() often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

Many Parameters

Methods with many parameters are not only hard to understand, but their parameters also often become inconsistent when you need more, or different data.

There are several approaches to avoid long parameter lists:

1
# SPDX-License-Identifier: LGPL-3.0-only
2
3
"""Representation of a collection of items."""
4
5
import os
6
import re
7
from collections import OrderedDict
8
from itertools import chain
9
from typing import Dict, List
10
11
import yaml
12
13
from doorstop import common, settings
14
from doorstop.common import DoorstopError, DoorstopInfo, DoorstopWarning
15
from doorstop.core.base import (
16
    BaseFileObject,
17
    BaseValidatable,
18
    add_document,
19
    auto_load,
20
    auto_save,
21
    delete_document,
22
    edit_document,
23
)
24
from doorstop.core.item import Item
25
from doorstop.core.types import UID, Level, Prefix
26
from doorstop.core.validators.item_validator import ItemValidator
27
28
log = common.logger(__name__)
29
30
31
class Document(BaseValidatable, BaseFileObject):  # pylint: disable=R0902
32
    """Represents a document directory containing an outline of items."""
33
34
    CONFIG = '.doorstop.yml'
35
    SKIP = '.doorstop.skip'  # indicates this document should be skipped
36
    ASSETS = 'assets'
37
    INDEX = 'index.yml'
38
39
    DEFAULT_PREFIX = Prefix('REQ')
40
    DEFAULT_SEP = ''
41
    DEFAULT_DIGITS = 3
42
43
    def __init__(self, path, root=os.getcwd(), **kwargs):
44
        """Initialize a document from an exiting directory.
45
46
        :param path: path to document directory
47
        :param root: path to root of project
48
49
        """
50
        super().__init__()
51
        # Ensure the directory is valid
52
        if not os.path.isfile(os.path.join(path, Document.CONFIG)):
53
            relpath = os.path.relpath(path, root)
54
            msg = "no {} in {}".format(Document.CONFIG, relpath)
55
            raise DoorstopError(msg)
56
        # Initialize the document
57
        self.path = path
58
        self.root = root
59
        self.tree = kwargs.get('tree')
60
        self.auto = kwargs.get('auto', Document.auto)
61
        # Set default values
62
        self._attribute_defaults = None
63
        self._attribute_publish = None
64
        self._data['prefix'] = Document.DEFAULT_PREFIX
65
        self._data['sep'] = Document.DEFAULT_SEP
66
        self._data['digits'] = Document.DEFAULT_DIGITS  # type: ignore
67
        self._data['parent'] = None  # type: ignore
68
        self._extended_reviewed: List[str] = []
69
        self._items: List[Item] = []
70
        self._itered = False
71
        self.children: List[Document] = []
72
73
    def __repr__(self):
74
        return "Document('{}')".format(self.path)
75
76
    def __str__(self):
77
        if common.verbosity < common.STR_VERBOSITY:
78
            return self.prefix
79
        else:
80
            return "{} ({})".format(self.prefix, self.relpath)
81
82
    def __iter__(self):
83
        yield from self._iter()
84
85
    def __len__(self):
86
        return len(list(i for i in self._iter() if i.active))
87
88
    def __bool__(self):
89
        """Even empty documents should be considered truthy."""
90
        return True
91
92
    @staticmethod
93
    @add_document
94
    def new(
95
        tree, path, root, prefix, sep=None, digits=None, parent=None, auto=None
96
    ):  # pylint: disable=R0913,C0301
97
        """Create a new document.
98
99
        :param tree: reference to tree that contains this document
100
101
        :param path: path to directory for the new document
102
        :param root: path to root of the project
103
        :param prefix: prefix for the new document
104
105
        :param sep: separator between prefix and numbers
106
        :param digits: number of digits for the new document
107
        :param parent: parent UID for the new document
108
        :param auto: automatically save the document
109
110
        :raises: :class:`~doorstop.common.DoorstopError` if the document
111
            already exists
112
113
        :return: new :class:`~doorstop.core.document.Document`
114
115
        """
116
        # Check separator
117
        if sep and sep not in settings.SEP_CHARS:
118
            raise DoorstopError("invalid UID separator '{}'".format(sep))
119
120
        config = os.path.join(path, Document.CONFIG)
121
122
        # Check for an existing document
123
        if os.path.exists(config):
124
            raise DoorstopError("document already exists: {}".format(path))
125
126
        # Create the document directory
127
        Document._create(config, name='document')
128
129
        # Initialize the document
130
        document = Document(path, root=root, tree=tree, auto=False)
131
        document.prefix = (  # type: ignore
132
            prefix if prefix is not None else document.prefix
133
        )
134
        document.sep = sep if sep is not None else document.sep  # type: ignore
135
        document.digits = (  # type: ignore
136
            digits if digits is not None else document.digits
137
        )
138
        document.parent = (  # type: ignore
139
            parent if parent is not None else document.parent
140
        )
141
        if auto or (auto is None and Document.auto):
142
            document.save()
143
144
        # Return the document
145
        return document
146
147
    def _load_with_include(self, yamlfile):
148
        """Load the YAML file and process input tags."""
149
        # Read text from file
150
        text = self._read(yamlfile)
151
        # Parse YAML data from text
152
        class IncludeLoader(yaml.SafeLoader):
153
            def include(self, node):
154
                container = IncludeLoader.filenames[0]  # type: ignore
155
                dirname = os.path.dirname(container)
156
                filename = os.path.join(dirname, self.construct_scalar(node))
157
                IncludeLoader.filenames.insert(0, filename)  # type: ignore
158
                try:
159
                    with open(filename, 'r') as f:
160
                        data = yaml.load(f, IncludeLoader)
161
                except Exception as ex:
162
                    msg = "include in '{}' failed: {}".format(container, ex)
163
                    raise DoorstopError(msg)
164
                IncludeLoader.filenames.pop()  # type: ignore
165
                return data
166
167
        IncludeLoader.add_constructor('!include', IncludeLoader.include)
168
        IncludeLoader.filenames = [yamlfile]  # type: ignore
169
        return self._load(text, yamlfile, loader=IncludeLoader)
170
171
    def load(self, reload=False):
172
        """Load the document's properties from its file."""
173
        if self._loaded and not reload:
174
            return
175
        log.debug("loading {}...".format(repr(self)))
176
        data = self._load_with_include(self.config)
177
        # Store parsed data
178
        sets = data.get('settings', {})
179
        for key, value in sets.items():
180
            try:
181
                if key == 'prefix':
182
                    self._data[key] = Prefix(value)
183
                elif key == 'sep':
184
                    self._data[key] = value.strip()
185
                elif key == 'parent':
186
                    self._data[key] = value.strip()
187
                elif key == 'digits':
188
                    self._data[key] = int(value)  # type: ignore
189
                else:
190
                    msg = "unexpected document setting '{}' in: {}".format(
191
                        key, self.config
192
                    )
193
                    raise DoorstopError(msg)
194
            except (AttributeError, TypeError, ValueError):
195
                msg = "invalid value for '{}' in: {}".format(key, self.config)
196
                raise DoorstopError(msg)
197
        # Store parsed attributes
198
        attributes = data.get('attributes', {})
199
        for key, value in attributes.items():
200
            if key == 'defaults':
201
                self._attribute_defaults = value
202
            elif key == 'reviewed':
203
                self._extended_reviewed = sorted(set(v for v in value))
204
            elif key == 'publish':
205
                self._attribute_publish = value
206
            else:
207
                msg = "unexpected attributes configuration '{}' in: {}".format(
208
                    key, self.config
209
                )
210
                raise DoorstopError(msg)
211
        # Set meta attributes
212
        self._loaded = True
213
        if reload:
214
            list(self._iter(reload=reload))
215
216
    @edit_document
217
    def save(self):
218
        """Save the document's properties to its file."""
219
        log.debug("saving {}...".format(repr(self)))
220
        # Format the data items
221
        data = {}
222
        sets = {}
223
        for key, value in self._data.items():
224
            if key == 'prefix':
225
                sets[key] = str(value)
226
            elif key == 'parent':
227
                if value:
228
                    sets[key] = value
229
            else:
230
                sets[key] = value
231
        data['settings'] = sets
232
        # Save the attributes
233
        attributes = {}
234
        if self._attribute_defaults:
235
            attributes['defaults'] = self._attribute_defaults
236
        if self._extended_reviewed:
237
            attributes['reviewed'] = self._extended_reviewed
238
        if attributes:
239
            data['attributes'] = attributes
240
        # Dump the data to YAML
241
        text = self._dump(data)
242
        # Save the YAML to file
243
        self._write(text, self.config)
244
        # Set meta attributes
245
        self._loaded = False
246
        self.auto = True
247
248
    def _iter(self, reload=False):
249
        """Yield the document's items."""
250
        if self._itered and not reload:
251
            msg = "iterating document {}'s loaded items...".format(self)
252
            log.debug(msg)
253
            yield from list(self._items)
254
            return
255
        log.info("loading document {}'s items...".format(self))
256
        # Reload the document's item
257
        self._items = []
258
        for dirpath, dirnames, filenames in os.walk(self.path):
259
            for dirname in list(dirnames):
260
                path = os.path.join(dirpath, dirname, Document.CONFIG)
261
                if os.path.exists(path):
262
                    path = os.path.dirname(path)
263
                    dirnames.remove(dirname)
264
                    log.trace(  # type: ignore
265
                        "skipped embedded document: {}".format(path)
266
                    )
267
            for filename in filenames:
268
                path = os.path.join(dirpath, filename)
269
                try:
270
                    item = Item(self, path, root=self.root, tree=self.tree)
271
                except DoorstopError:
272
                    pass  # skip non-item files
273
                else:
274
                    self._items.append(item)
275
                    if reload:
276
                        try:
277
                            item.load(reload=reload)
278
                        except Exception:
279
                            log.error("Unable to load: %s", item)
280
                            raise
281
                    if settings.CACHE_ITEMS and self.tree:
282
                        self.tree._item_cache[  # pylint: disable=protected-access
283
                            item.uid
284
                        ] = item
285
                        log.trace("cached item: {}".format(item))  # type: ignore
286
        # Set meta attributes
287
        self._itered = True
288
        # Yield items
289
        yield from list(self._items)
290
291
    def copy_assets(self, dest):
292
        """Copy the contents of the assets directory."""
293
        if not self.assets:
294
            return
295
        common.copy_dir_contents(self.assets, dest)
296
297
    # properties #############################################################
298
299
    @property
300
    def config(self):
301
        """Get the path to the document's file."""
302
        assert self.path
303
        return os.path.join(self.path, Document.CONFIG)
304
305
    @property
306
    def assets(self):
307
        """Get the path to the document's assets if they exist else `None`."""
308
        assert self.path
309
        path = os.path.join(self.path, Document.ASSETS)
310
        return path if os.path.isdir(path) else None
311
312
    @property  # type: ignore
313
    @auto_load
314
    def prefix(self):
315
        """Get the document's prefix."""
316
        return self._data['prefix']
317
318
    @property  # type: ignore
319
    @auto_load
320
    def publish(self):
321
        """Get the document's prefix."""
322
        return self._attribute_publish
323
324
    @prefix.setter  # type: ignore
325
    @auto_save
326
    @auto_load
327
    def prefix(self, value):
328
        """Set the document's prefix."""
329
        self._data['prefix'] = Prefix(value)
330
        # TODO: should the new prefix be applied to all items?
331
332
    @property  # type: ignore
333
    @auto_load
334
    def extended_reviewed(self):
335
        """Get the document's extended reviewed attribute keys."""
336
        return self._extended_reviewed
337
338
    @property  # type: ignore
339
    @auto_load
340
    def sep(self):
341
        """Get the prefix-number separator to use for new item UIDs."""
342
        return self._data['sep']
343
344
    @sep.setter  # type: ignore
345
    @auto_save
346
    @auto_load
347
    def sep(self, value):
348
        """Set the prefix-number separator to use for new item UIDs."""
349
        # TODO: raise a specific exception for invalid separator characters?
350
        assert not value or value in settings.SEP_CHARS
351
        self._data['sep'] = value.strip()
352
        # TODO: should the new separator be applied to all items?
353
354
    @property  # type: ignore
355
    @auto_load
356
    def digits(self):
357
        """Get the number of digits to use for new item UIDs."""
358
        return self._data['digits']
359
360
    @digits.setter  # type: ignore
361
    @auto_save
362
    @auto_load
363
    def digits(self, value):
364
        """Set the number of digits to use for new item UIDs."""
365
        self._data['digits'] = value
366
        # TODO: should the new digits be applied to all items?
367
368
    @property  # type: ignore
369
    @auto_load
370
    def parent(self):
371
        """Get the document's parent document prefix."""
372
        return self._data['parent']
373
374
    @parent.setter  # type: ignore
375
    @auto_save
376
    @auto_load
377
    def parent(self, value):
378
        """Set the document's parent document prefix."""
379
        self._data['parent'] = str(value) if value else ""
380
381
    @property
382
    def items(self):
383
        """Get an ordered list of active items in the document."""
384
        return sorted(i for i in self._iter() if i.active)
385
386
    @property
387
    def depth(self):
388
        """Return the maximum item level depth."""
389
        return max(item.depth for item in self)
390
391
    @property
392
    def next_number(self):
393
        """Get the next item number for the document."""
394
        try:
395
            number = max(item.uid.number for item in self) + 1
396
        except ValueError:
397
            number = 1
398
        log.debug("next number (local): {}".format(number))
399
400
        if self.tree and self.tree.request_next_number:
401
            remote_number = 0
402
            while remote_number is not None and remote_number < number:
403
                if remote_number:
404
                    log.warning("server is behind, requesting next number...")
405
                remote_number = self.tree.request_next_number(self.prefix)
406
                log.debug("next number (remote): {}".format(remote_number))
407
            if remote_number:
408
                number = remote_number
409
410
        return number
411
412
    @property
413
    def skip(self):
414
        """Indicate the document should be skipped."""
415
        assert self.path
416
        return os.path.isfile(os.path.join(self.path, Document.SKIP))
417
418
    @property
419
    def index(self):
420
        """Get the path to the document's index if it exists else `None`."""
421
        assert self.path
422
        path = os.path.join(self.path, Document.INDEX)
423
        return path if os.path.isfile(path) else None
424
425
    @index.setter
426
    def index(self, value):
427
        """Create or update the document's index."""
428
        if value:
429
            assert self.path
430
            path = os.path.join(self.path, Document.INDEX)
431
            log.info("creating {} index...".format(self))
432
            common.write_lines(self._lines_index(self.items), path)
433
434
    @index.deleter
435
    def index(self):
436
        """Delete the document's index if it exists."""
437
        log.info("deleting {} index...".format(self))
438
        common.delete(self.index)
439
440
    # actions ################################################################
441
442
    # decorators are applied to methods in the associated classes
443
    def add_item(self, number=None, level=None, reorder=True, defaults=None, name=None):
444
        """Create a new item for the document and return it.
445
446
        :param number: desired item number
447
        :param level: desired item level
448
        :param reorder: update levels of document items
449
450
        :return: added :class:`~doorstop.core.item.Item`
451
452
        """
453
        uid = None
454
        if name is None:
455
            number = max(number or 0, self.next_number)
456
            log.debug("next number: {}".format(number))
457
            uid = UID(self.prefix, self.sep, number, self.digits)
458
        else:
459
            try:
460
                uid = UID(self.prefix, self.sep, int(name), self.digits)
461
            except ValueError:
462
                if not self.sep:
463
                    msg = "cannot add item with name '{}' to document '{}' without a separator".format(
464
                        name, self.prefix
465
                    )
466
                    raise DoorstopError(msg)
467
                if self.sep not in settings.SEP_CHARS:
468
                    msg = "cannot add item with name '{}' to document '{}' with an invalid separator '{}'".format(
469
                        name, self.prefix, self.sep
470
                    )
471
                    raise DoorstopError(msg)
472
                uid = UID(self.prefix, self.sep, name)
473
                if uid.prefix != self.prefix or uid.name != name:
474
                    msg = "invalid item name '{}'".format(name)
475
                    raise DoorstopError(msg)
476
477
        try:
478
            last = self.items[-1]
479
        except IndexError:
480
            next_level = level
481
        else:
482
            if level:
483
                next_level = level
484
            elif last.level.heading:
485
                next_level = last.level >> 1
486
                next_level.heading = False
487
            else:
488
                next_level = last.level + 1
489
        log.debug("next level: {}".format(next_level))
490
491
        # Load more defaults before the item is created to avoid partially
492
        # constructed items in case the loading fails.
493
        more_defaults = self._load_with_include(defaults) if defaults else None
494
495
        item = Item.new(self.tree, self, self.path, self.root, uid, level=next_level)
496
        if self._attribute_defaults:
497
            item.set_attributes(self._attribute_defaults)
498
        if more_defaults:
499
            item.set_attributes(more_defaults)
500
        if level and reorder:
501
            self.reorder(keep=item)
502
        return item
503
504
    # decorators are applied to methods in the associated classes
505
    def remove_item(self, value, reorder=True):
506
        """Remove an item by its UID.
507
508
        :param value: item or UID
509
        :param reorder: update levels of document items
510
511
        :raises: :class:`~doorstop.common.DoorstopError` if the item
512
            cannot be found
513
514
        :return: removed :class:`~doorstop.core.item.Item`
515
516
        """
517
        uid = UID(value)
518
        item = self.find_item(uid)
519
        item.delete()
520
        if reorder:
521
            self.reorder()
522
        return item
523
524
    # decorators are applied to methods in the associated classes
525
    def reorder(self, manual=True, automatic=True, start=None, keep=None, _items=None):
526
        """Reorder a document's items.
527
528
        Two methods are using to create the outline order:
529
530
        - manual: specify the order using an updated index file
531
        - automatic: shift duplicate levels and compress gaps
532
533
        :param manual: enable manual ordering using the index (if one exists)
534
535
        :param automatic: enable automatic ordering (after manual ordering)
536
        :param start: level to start numbering (None = use current start)
537
        :param keep: item or UID to keep over duplicates
538
539
        """
540
        # Reorder manually
541
        if manual and self.index:
542
            log.info("reordering {} from index...".format(self))
543
            self._reorder_from_index(self, self.index)
544
            del self.index
545
        # Reorder automatically
546
        if automatic:
547
            log.info("reordering {} automatically...".format(self))
548
            items = _items or self.items
549
            keep = self.find_item(keep) if keep else None
550
            self._reorder_automatic(items, start=start, keep=keep)
551
552
    @staticmethod
553
    def _lines_index(items):
554
        """Generate (pseudo) YAML lines for the document index."""
555
        yield '#' * settings.MAX_LINE_LENGTH
556
        yield '# THIS TEMPORARY FILE WILL BE DELETED AFTER DOCUMENT REORDERING'
557
        yield '# MANUALLY INDENT, DEDENT, & MOVE ITEMS TO THEIR DESIRED LEVEL'
558
        yield '# A NEW ITEM WILL BE ADDED FOR ANY UNKNOWN IDS, i.e. - new: '
559
        yield '# THE COMMENT WILL BE USED AS THE ITEM TEXT FOR NEW ITEMS'
560
        yield '# CHANGES WILL BE REFLECTED IN THE ITEM FILES AFTER CONFIRMATION'
561
        yield '#' * settings.MAX_LINE_LENGTH
562
        yield ''
563
        yield "initial: {}".format(items[0].level if items else 1.0)
564
        yield "outline:"
565
        for item in items:
566
            space = "    " * item.depth
567
            lines = item.text.strip().splitlines()
568
            comment = lines[0] if lines else ""
569
            line = space + "- {u}: # {c}".format(u=item.uid, c=comment)
570
            if len(line) > settings.MAX_LINE_LENGTH:
571
                line = line[: settings.MAX_LINE_LENGTH - 3] + '...'
572
            yield line
573
574
    @staticmethod
575
    def _read_index(path):
576
        """Load the index, converting comments to text entries for each item."""
577
        with open(path, 'r', encoding='utf-8') as stream:
578
            text = stream.read()
579
        yaml_text = []
580
        for line in text.split('\n'):
581
            m = re.search(r'(\s+)(- [\w\d-]+\s*): # (.+)$', line)
582
            if m:
583
                prefix = m.group(1)
584
                uid = m.group(2)
585
                item_text = m.group(3).replace('"', '\\"')
586
                yaml_text.append('{p}{u}:'.format(p=prefix, u=uid))
587
                yaml_text.append('    {p}- text: "{t}"'.format(p=prefix, t=item_text))
588
            else:
589
                yaml_text.append(line)
590
        return common.load_yaml('\n'.join(yaml_text), path)
591
592
    @staticmethod
593
    def _reorder_from_index(document, path):
594
        """Reorder a document's item from the index."""
595
        data = document._read_index(path)  # pylint: disable=protected-access
596
        # Read updated values
597
        initial = data.get('initial', 1.0)
598
        outline = data.get('outline', [])
599
        # Update levels
600
        level = Level(initial)
601
        ids_after_reorder: List[str] = []
602
        Document._reorder_section(outline, level, document, ids_after_reorder)
603
        for item in document.items:
604
            if item.uid not in ids_after_reorder:
605
                log.info('Deleting %s', item.uid)
606
                item.delete()
607
608
    @staticmethod
609
    def _reorder_section(section, level, document, list_of_ids):
610
        """Recursive function to reorder a section of an outline.
611
612
        :param section: recursive `list` of `dict` loaded from document index
613
        :param level: current :class:`~doorstop.core.types.Level`
614
        :param document: :class:`~doorstop.core.document.Document` to order
615
616
        """
617
        if isinstance(section, dict):  # a section
618
619
            # Get the item and subsection
620
            uid = list(section.keys())[0]
621
            if uid == 'text':
622
                return
623
            subsection = section[uid]
624
625
            # An item is a header if it has a subsection
626
            level.heading = False
627
            item_text = ''
628
            if isinstance(subsection, str):
629
                item_text = subsection
630
            elif isinstance(subsection, list):
631
                if 'text' in subsection[0]:
632
                    item_text = subsection[0]['text']
633
                    if len(subsection) > 1:
634
                        level.heading = True
635
636
            try:
637
                item = document.find_item(uid)
638
                item.level = level
639
                log.info("Found ({}): {}".format(uid, level))
640
                list_of_ids.append(uid)
641
            except DoorstopError:
642
                item = document.add_item(level=level, reorder=False)
643
                list_of_ids.append(item.uid)
644
                if level.heading:
645
                    item.normative = False
646
                item.text = item_text
647
                log.info("Created ({}): {}".format(item.uid, level))
648
649
            # Process the heading's subsection
650
            if subsection:
651
                Document._reorder_section(subsection, level >> 1, document, list_of_ids)
652
653
        elif isinstance(section, list):  # a list of sections
654
655
            # Process each subsection
656
            for index, subsection in enumerate(section):
657
                Document._reorder_section(
658
                    subsection, level + index, document, list_of_ids
659
                )
660
661
    @staticmethod
662
    def _reorder_automatic(items, start=None, keep=None):
663
        """Reorder a document's items automatically.
664
665
        :param items: items to reorder
666
        :param start: level to start numbering (None = use current start)
667
        :param keep: item to keep over duplicates
668
669
        """
670
        nlevel = plevel = None
671
        for clevel, item in Document._items_by_level(items, keep=keep):
672
            log.debug("current level: {}".format(clevel))
673
            # Determine the next level
674
            if not nlevel:
675
                # Use the specified or current starting level
676
                nlevel = Level(start) if start else clevel
677
                nlevel.heading = clevel.heading
678
                log.debug("next level (start): {}".format(nlevel))
679
            else:
680
                # Adjust the next level to be the same depth
681
                if len(clevel) > len(nlevel):
682
                    nlevel >>= len(clevel) - len(nlevel)
683
                    log.debug("matched current indent: {}".format(nlevel))
684
                elif len(clevel) < len(nlevel):
685
                    nlevel <<= len(nlevel) - len(clevel)
686
                    # nlevel += 1
687
                    log.debug("matched current dedent: {}".format(nlevel))
688
                nlevel.heading = clevel.heading
689
                # Check for a level jump
690
                _size = min(len(clevel.value), len(plevel.value))
691
                for index in range(max(_size - 1, 1)):
692
                    if clevel.value[index] > plevel.value[index]:
693
                        nlevel <<= len(nlevel) - 1 - index
694
                        nlevel += 1
695
                        nlevel >>= len(clevel) - len(nlevel)
696
                        msg = "next level (jump): {}".format(nlevel)
697
                        log.debug(msg)
698
                        break
699
                # Check for a normal increment
700
                else:
701
                    if len(nlevel) <= len(plevel):
702
                        nlevel += 1
703
                        msg = "next level (increment): {}".format(nlevel)
704
                        log.debug(msg)
705
                    else:
706
                        msg = "next level (indent/dedent): {}".format(nlevel)
707
                        log.debug(msg)
708
            # Apply the next level
709
            if clevel == nlevel:
710
                log.info("{}: {}".format(item, clevel))
711
            else:
712
                log.info("{}: {} to {}".format(item, clevel, nlevel))
713
            item.level = nlevel.copy()
714
            # Save the current level as the previous level
715
            plevel = clevel.copy()
716
717
    @staticmethod
718
    def _items_by_level(items, keep=None):
719
        """Iterate through items by level with the kept item first."""
720
        # Collect levels
721
        levels: Dict[Level, List[Item]] = OrderedDict()
722
        for item in items:
723
            if item.level in levels:
724
                levels[item.level].append(item)
725
            else:
726
                levels[item.level] = [item]
727
        # Reorder levels
728
        for level, items_at_level in levels.items():
729
            # Reorder items at this level
730
            if keep in items_at_level:
731
                # move the kept item to the front of the list
732
                log.debug("keeping {} level over duplicates".format(keep))
733
                items_at_level.remove(keep)
734
                items_at_level.insert(0, keep)
735
            for item in items_at_level:
736
                yield level, item
737
738
    def find_item(self, value, _kind=''):
739
        """Return an item by its UID.
740
741
        :param value: item or UID
742
743
        :raises: :class:`~doorstop.common.DoorstopError` if the item
744
            cannot be found
745
746
        :return: matching :class:`~doorstop.core.item.Item`
747
748
        """
749
        uid = UID(value)
750
        for item in self:
751
            if item.uid == uid:
752
                if item.active:
753
                    return item
754
                else:
755
                    log.trace("item is inactive: {}".format(item))  # type: ignore
756
757
        raise DoorstopError("no matching{} UID: {}".format(_kind, uid))
758
759
    def get_issues(
760
        self, skip=None, document_hook=None, item_hook=None
761
    ):  # pylint: disable=unused-argument
762
        """Yield all the document's issues.
763
764
        :param skip: list of document prefixes to skip
765
        :param item_hook: function to call for custom item validation
766
767
        :return: generator of :class:`~doorstop.common.DoorstopError`,
768
                              :class:`~doorstop.common.DoorstopWarning`,
769
                              :class:`~doorstop.common.DoorstopInfo`
770
771
        """
772
        assert document_hook is None
773
        skip = [] if skip is None else skip
774
        hook = item_hook if item_hook else lambda **kwargs: []
775
776
        if self.prefix in skip:
777
            log.info("skipping document %s...", self)
778
            return
779
        else:
780
            log.info("checking document %s...", self)
781
782
        # Check for items
783
        items = self.items
784
        if not items:
785
            yield DoorstopWarning("no items")
786
            return
787
788
        # Reorder or check item levels
789
        if settings.REORDER:
790
            self.reorder(_items=items)
791
        elif settings.CHECK_LEVELS:
792
            yield from self._get_issues_level(items)
793
794
        item_validator = ItemValidator()
795
796
        # Check each item
797
        for item in items:
798
799
            # Check item
800
            for issue in chain(
801
                hook(item=item, document=self, tree=self.tree),
802
                item_validator.get_issues(item, skip=skip),
803
            ):
804
805
                # Prepend the item's UID to yielded exceptions
806
                if isinstance(issue, Exception):
807
                    yield type(issue)("{}: {}".format(item.uid, issue))
808
809
    @staticmethod
810
    def _get_issues_level(items):
811
        """Yield all the document's issues related to item level."""
812
        prev = items[0] if items else None
813
        for item in items[1:]:
814
            puid = prev.uid
815
            plev = prev.level
816
            nuid = item.uid
817
            nlev = item.level
818
            log.debug("checking level {} to {}...".format(plev, nlev))
819
            # Duplicate level
820
            if plev == nlev:
821
                uids = sorted((puid, nuid))
822
                msg = "duplicate level: {} ({}, {})".format(plev, *uids)
823
                yield DoorstopWarning(msg)
824
            # Skipped level
825
            length = min(len(plev.value), len(nlev.value))
826
            for index in range(length):
827
                # Types of skipped levels:
828
                #         1. over: 1.0 --> 1.2
829
                #         2. out: 1.1 --> 3.0
830
                if (
831
                    nlev.value[index] - plev.value[index] > 1
832
                    or
833
                    # 3. over and out: 1.1 --> 2.2
834
                    (
835
                        plev.value[index] != nlev.value[index]
836
                        and index + 1 < length
837
                        and nlev.value[index + 1] not in (0, 1)
838
                    )
839
                ):
840
                    msg = "skipped level: {} ({}), {} ({})".format(
841
                        plev, puid, nlev, nuid
842
                    )
843
                    yield DoorstopInfo(msg)
844
                    break
845
            prev = item
846
847
    @delete_document
848
    def delete(self, path=None):
849
        """Delete the document and its items."""
850
        for item in self:
851
            item.delete()
852
        # the document is deleted in the decorated method
853