bbarchivist.scripts.archivist.archivist_main() - Code Metrics - thurask/bbarchivist - Measure and Improve Code Quality continuously with Scrutinizer

bbarchivist.scripts.archivist.archivist_main() D
last analyzed 2019-11-17 11:32 UTC

↳ Parent: bbarchivist.scripts.archivist

Complexity

Conditions

Size

Total Lines	122
Code Lines	59

Duplication

Lines	0
Ratio	0 %

Importance

Changes

Metric	Value
cc	12
eloc	59
nop	18
dl	0
loc	122
rs	4.7018
c	0
b	0
f	0

How to fix Long Method Complexity Many Parameters

Long Method

Small methods make your code easier to understand, in particular if combined with a good name. Besides, if your method is small, finding a good name is usually much easier.

For example, if you find yourself adding comments to a method's body, this is usually a good sign to extract the commented part to a new method, and use the comment as a starting point when coming up with a good name for this new method.

Commonly applied refactorings include:

If many parameters/temporary variables are present:
- Replace temporary variables with Query
- Introduce parameter object; often combined with preserve whole object
- If the above two are insufficient: Replace method with method object
If you have long conditionals: Decompose Conditional
Otherwise: Extract method

Complexity

Complex classes like bbarchivist.scripts.archivist.archivist_main() often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

Many Parameters

Methods with many parameters are not only hard to understand, but their parameters also often become inconsistent when you need more, or different data.

There are several approaches to avoid long parameter lists:

If you can get the data from another object that the method knows about already: Replace the parameter with method call
If several parameters are part of another object: Preserve Whole Object
If parameters are not yet connected: Introduce Parameter Object

#!/usr/bin/env python3
"""Download bar files, create autoloaders."""

import os  # filesystem read
import sys  # load arguments

import requests  # session
from bbarchivist import archiveutils  # archive work
from bbarchivist import argutils  # arguments
from bbarchivist import barutils  # file/folder work
from bbarchivist import decorators  # timer
from bbarchivist import hashutils  # hashes, GPG
from bbarchivist import loadergen  # cap, in Python
from bbarchivist import networkutils  # download/lookup
from bbarchivist import scriptutils  # script stuff
from bbarchivist import utilities  # input validation

__author__ = "Thurask"
__license__ = "WTFPL v2"
__copyright__ = "2015-2019 Thurask"


@decorators.timer
def grab_args():
    """
    Parse arguments from argparse/questionnaire.

    Invoke :func:`archivist.archivist_main` with those arguments.
    """
    if len(sys.argv) > 1:
        parser = argutils.default_parser("bb-archivist", "Create autoloaders", ("folder", "osr"))
        negategroup = parser.add_argument_group(
            "negators",
            "Disable program functionality")
        negategroup.add_argument(
            "-no",
            "--no-download",
            dest="download",
            help="Don't download files",
            action="store_false",
            default=True)
        negategroup.add_argument(
            "-ni",
            "--no-integrity",
            dest="integrity",
            help="Don't test bar files after download",
            action="store_false",
            default=True)
        negategroup.add_argument(
            "-nx",
            "--no-extract",
            dest="extract",
            help="Don't extract bar files",
            action="store_false",
            default=True)
        negategroup.add_argument(
            "-nr",
            "--no-radios",
            dest="radloaders",
            help="Don't make radio autoloaders",
            action="store_false",
            default=True)
        negategroup.add_argument(
            "-ns",
            "--no-rmsigned",
            dest="signed",
            help="Don't remove signed files",
            action="store_false",
            default=True)
        negategroup.add_argument(
            "-nc",
            "--no-compress",
            dest="compress",
            help="Don't compress loaders",
            action="store_false",
            default=True)
        negategroup.add_argument(
            "-nd",
            "--no-delete",
            dest="delete",
            help="Don't delete uncompressed loaders",
            action="store_false",
            default=True)
        negategroup.add_argument(
            "-nv",
            "--no-verify",
            dest="verify",
            help="Don't verify created loaders",
            action="store_false",
            default=True)
        if not getattr(sys, 'frozen', False):
            parser.add_argument(
                "-g",
                "--gpg",
                dest="gpg",
                help="Enable GPG signing. Set up GnuPG.",
                action="store_true",
                default=False)
        parser.add_argument(
            "-r",
            "--radiosw",
            dest="altsw",
            metavar="SW",
            help="Radio software version; use without software to guess",
            nargs="?",
            const="checkme",
            default=None)
        if not getattr(sys, 'frozen', False):
            parser.add_argument(
                "-m",
                "--method",
                dest="method",
                metavar="METHOD",
                help="Compression method",
                nargs="?",
                type=argutils.valid_method,
                default=None)
        parser.add_argument(
            "-c",
            "--core",
            dest="core",
            help="Make core/radio loaders",
            default=False,
            action="store_true")
        parser.add_argument(
            "-o",
            "--old-style",
            dest="oldstyle",
            help="Make old-style checksum files",
            default=False,
            action="store_true")
        parser.set_defaults(compmethod="7z")
        args = parser.parse_args(sys.argv[1:])
        args.folder = scriptutils.generate_workfolder(args.folder)
        if getattr(sys, 'frozen', False):
            args.gpg = False
            hashdict = hashutils.verifier_config_loader(os.getcwd())
            args.method = "7z"
        else:
            hashdict = hashutils.verifier_config_loader()
        hashutils.verifier_config_writer(hashdict)
        if args.method is None:
            compmethod = archiveutils.compress_config_loader()
            archiveutils.compress_config_writer(compmethod)
        else:
            compmethod = args.method
        archivist_main(args.os, args.radio, args.swrelease,
                       os.path.abspath(args.folder), args.radloaders,
                       args.compress, args.delete, args.verify,
                       hashdict, args.download,
                       args.extract, args.signed, compmethod, args.gpg,
                       args.integrity, args.altsw, args.core, args.oldstyle)
    else:
        questionnaire()


def questionnaire():
    """
    Questions to ask if no arguments given.
    """
    localdir = os.getcwd()
    osversion = input("OS VERSION (REQUIRED): ")
    radioversion = input("RADIO VERSION (PRESS ENTER TO GUESS): ")
    radioversion = None if not radioversion else radioversion
    softwareversion = input("OS SOFTWARE RELEASE (PRESS ENTER TO GUESS): ")
    softwareversion = None if not softwareversion else softwareversion
    altcheck = utilities.i2b("USING ALTERNATE RADIO (Y/N)?: ")
    altsw = scriptutils.check_altsw(altcheck)
    radios = utilities.i2b("CREATE RADIO LOADERS (Y/N)?: ")
    compressed = utilities.i2b("COMPRESS LOADERS (Y/N)?: ")
    deleted = utilities.i2b("DELETE UNCOMPRESSED LOADERS (Y/N)?: ") if compressed else False
    hashed = utilities.i2b("GENERATE HASHES (Y/N)?: ")
    if getattr(sys, 'frozen', False):
        hashdict = hashutils.verifier_config_loader(os.getcwd())
        compmethod = "7z"
    else:
        hashdict = hashutils.verifier_config_loader()
        hashutils.verifier_config_writer(hashdict)
        compmethod = archiveutils.compress_config_loader()
    print(" ")
    archivist_main(osversion, radioversion, softwareversion,
                   localdir, radios, compressed, deleted, hashed,
                   hashdict, download=True, extract=True, signed=True,
                   compmethod=compmethod, gpg=False, integrity=True,
                   altsw=altsw, core=False, oldstyle=False)


def archivist_checksw(baseurl, softwareversion, swchecked):
    """
    Check availability of software releases.

    :param baseurl: Base URL for download links.
    :type baseurl: str

    :param softwareversion: Software release, 10.x.y.zzzz. Can be guessed.
    :type softwareversion: str

    :param swchecked: If we checked the sw release already.
    :type swchecked: bool
    """
    scriptutils.check_sw(baseurl, softwareversion, swchecked)


def archivist_download(download, osurls, radiourls, localdir, session, dirs):
    """
    Download function.

    :param download: Whether to download bar files. True by default.
    :type download: bool

    :param osurls: OS file list.
    :type osurls: list(str)

    :param radiourls: Radio file list.
    :type radiourls: list(str)

    :param localdir: Working directory. Local by default.
    :type localdir: str

    :param session: Requests session object, default is created on the fly.
    :type session: requests.Session()

    :param dirs: List of generated bar/loader/zip directories.
    :type dirs: list(str)
    """
    osfiles = scriptutils.comp_joiner(localdir, dirs[0], osurls)
    radiofiles = scriptutils.comp_joiner(localdir, dirs[1], radiourls)
    if download:
        print("BEGIN DOWNLOADING...")
        networkutils.download_bootstrap(radiourls + osurls, localdir, 3, session)
        print("ALL FILES DOWNLOADED")
    elif all(os.path.exists(x) for x in osfiles+radiofiles):
        print("USING CACHED OS/RADIO FILES...")
        barutils.replace_bars_bulk(os.path.abspath(localdir), osfiles+radiofiles)


def archivist_integritybars(integrity, osurls, radiourls, localdir):
    """
    Check integrity of bar files, redownload if necessary.

    :param integrity: Whether to test downloaded files. True by default.
    :type integrity: bool

    :param osurls: OS file list.
    :type osurls: list(str)

    :param radiourls: Radio file list.
    :type radiourls: list(str)
    """
    if integrity:
        urllist = osurls + radiourls
        scriptutils.test_bar_files(localdir, urllist)


def archivist_extractbars(extract, localdir):
    """
    Extract signed files from bar files.

    :param extract: Whether to extract bar files. True by default.
    :type extract: bool

    :param localdir: Working directory. Local by default.
    :type localdir: str
    """
    if extract:
        print("EXTRACTING...")
        barutils.extract_bars(localdir)


def archivist_integritysigned(integrity, localdir):
    """
    Check integrity of signed files.

    :param integrity: Whether to test downloaded files. True by default.
    :type integrity: bool

    :param localdir: Working directory. Local by default.
    :type localdir: str
    """
    if integrity:
        scriptutils.test_signed_files(localdir)


def archivist_movebars(dirs, localdir):
    """
    Move bar files.

    :param dirs: List of OS/radio bar/loader/zipped folders.
    :type dirs: list(str)

    :param localdir: Working directory. Local by default.
    :type localdir: str
    """
    print("MOVING BAR FILES...")
    barutils.move_bars(localdir, dirs[0], dirs[1])


def archivist_generateloaders(osversion, radioversion, radios, localdir, altsw, core):
    """
    Generate loaders.

    :param osversion: OS version, 10.x.y.zzzz. Required.
    :type osversion: str

    :param radioversion: Radio version, 10.x.y.zzzz. Can be guessed.
    :type radioversion: str

    :param radios: Whether to create radio autoloaders. True by default.
    :type radios: bool

    :param localdir: Working directory. Local by default.
    :type localdir: str

    :param altsw: Radio software release, if not the same as OS.
    :type altsw: str

    :param core: Whether to create a core/radio loader. Default is false.
    :type core: bool
    """
    print("GENERATING LOADERS...")
    altradio = altsw is not None
    loadergen.generate_loaders(osversion, radioversion, radios, localdir, altradio, core)


def archivist_integrityloaders(integrity, localdir):
    """
    Check integrity of build loaders.

    :param integrity: Whether to test downloaded files. True by default.
    :type integrity: bool

    :param localdir: Working directory. Local by default.
    :type localdir: str
    """
    if integrity:
        scriptutils.test_loader_files(localdir)


def archivist_removesigned(signed, localdir):
    """
    Remove signed files.

    :param signed: Whether to delete signed files. True by default.
    :type signed: bool

    :param localdir: Working directory. Local by default.
    :type localdir: str
    """
    if signed:
        print("REMOVING SIGNED FILES...")
        barutils.remove_signed_files(localdir)


def archivist_compressor(compressed, integrity, localdir, compmethod, szexe):
    """
    Compress and optionally verify loaders.

    :param compressed: Whether to compress files. True by default.
    :type compressed: bool

    :param integrity: Whether to test downloaded files. True by default.
    :type integrity: bool

    :param localdir: Working directory. Local by default.
    :type localdir: str

    :param compmethod: Compression method. Default is "7z", fallback "zip".
    :type compmethod: str

    :param szexe: Path to 7z executable.
    :type szexe: str
    """
    if compressed:
        print("COMPRESSING...")
        archiveutils.compress(localdir, compmethod, szexe, True)
        if integrity:
            print("TESTING ARCHIVES...")
            archiveutils.verify(localdir, compmethod, szexe, True)


def archivist_moveloaders(dirs, localdir):
    """
    Move loaders.

    :param dirs: List of OS/radio bar/loader/zipped folders.
    :type dirs: list(str)

    :param localdir: Working directory. Local by default.
    :type localdir: str
    """
    print("MOVING LOADERS...")
    barutils.move_loaders(localdir, dirs[2], dirs[3], dirs[4], dirs[5])


def archivist_gethashes(dirs, hashed, compressed, deleted, radios, osversion, radioversion, softwareversion, oldstyle):
    """
    Make new-style info files.

    :param dirs: List of OS/radio bar/loader/zipped folders.
    :type dirs: list(str)

    :param hashed: Whether to hash files. True by default.
    :type hashed: bool

    :param compressed: Whether to compress files. True by default.
    :type compressed: bool

    :param deleted: Whether to delete uncompressed files. True by default.
    :type deleted: bool

    :param radios: Whether to create radio autoloaders. True by default.
    :type radios: bool

    :param osversion: OS version, 10.x.y.zzzz. Required.
    :type osversion: str

    :param radioversion: Radio version, 10.x.y.zzzz. Can be guessed.
    :type radioversion: str

    :param softwareversion: Software release, 10.x.y.zzzz. Can be guessed.
    :type softwareversion: str

    :param oldstyle: Whether to make old-style checksum files. Default is false.
    :type oldstyle: bool
    """
    if hashed and not oldstyle:
        scriptutils.bulk_info(dirs, osversion, compressed, deleted, radios, radioversion, softwareversion)

def archivist_getoldhashes(dirs, hashed, compressed, deleted, radios, hashdict, oldstyle):
    """
    Make old-style checksum files.

    :param dirs: List of OS/radio bar/loader/zipped folders.
    :type dirs: list(str)

    :param hashed: Whether to hash files. True by default.
    :type hashed: bool

    :param compressed: Whether to compress files. True by default.
    :type compressed: bool

    :param deleted: Whether to delete uncompressed files. True by default.
    :type deleted: bool

    :param radios: Whether to create radio autoloaders. True by default.
    :type radios: bool

    :param hashdict: Dictionary of hash rules, in ~\bbarchivist.ini.
    :type hashdict: dict({str: bool})

    :param oldstyle: Whether to make old-style checksum files. Default is false.
    :type oldstyle: bool
    """
    if hashed and oldstyle:
        scriptutils.bulk_hash(dirs, compressed, deleted, radios, hashdict)


def archivist_gpg(gpg, dirs, compressed, deleted, radios):
    """
    GPG-sign everything.

    :param gpg: Whether to use GnuPG verification. False by default.
    :type gpg: bool

    :param dirs: List of OS/radio bar/loader/zipped folders.
    :type dirs: list(str)

    :param compressed: Whether to compress files. True by default.
    :type compressed: bool

    :param deleted: Whether to delete uncompressed files. True by default.
    :type deleted: bool

    :param radios: Whether to create radio autoloaders. True by default.
    :type radios: bool
    """
    if gpg:
        scriptutils.bulk_verify(dirs, compressed, deleted, radios)


def archivist_deleteuncompressed(dirs, deleted, radios):
    """
    Delete uncompressed loaders.

    :param dirs: List of OS/radio bar/loader/zipped folders.
    :type dirs: list(str)

    :param deleted: Whether to delete uncompressed files. True by default.
    :type deleted: bool

    :param radios: Whether to create radio autoloaders. True by default.
    :type radios: bool
    """
    if deleted:
        print("DELETING UNCOMPRESSED LOADERS...")
        barutils.remove_unpacked_loaders(dirs[2], dirs[3], radios)


def archivist_removeemptyfolders(localdir):
    """
    Delete empty folders.

    :param localdir: Working directory. Local by default.
    :type localdir: str
    """
    print("REMOVING EMPTY FOLDERS...")
    barutils.remove_empty_folders(localdir)


def archivist_main(osversion, radioversion=None, softwareversion=None,
                   localdir=None, radios=True, compressed=True, deleted=True,
                   hashed=True, hashdict=None, download=True,
                   extract=True, signed=True, compmethod="7z",
                   gpg=False, integrity=True, altsw=None,
                   core=False, oldstyle=False):
    """
    Wrap around multi-autoloader creation code.
    Some combination of creating, downloading, hashing,
    compressing and moving autoloaders.

    :param osversion: OS version, 10.x.y.zzzz. Required.
    :type osversion: str

    :param radioversion: Radio version, 10.x.y.zzzz. Can be guessed.
    :type radioversion: str

    :param softwareversion: Software release, 10.x.y.zzzz. Can be guessed.
    :type softwareversion: str

    :param localdir: Working directory. Local by default.
    :type localdir: str

    :param radios: Whether to create radio autoloaders. True by default.
    :type radios: bool

    :param compressed: Whether to compress files. True by default.
    :type compressed: bool

    :param deleted: Whether to delete uncompressed files. True by default.
    :type deleted: bool

    :param hashed: Whether to hash files. True by default.
    :type hashed: bool

    :param hashdict: Dictionary of hash rules, in ~\bbarchivist.ini.
    :type hashdict: dict({str: bool})

    :param download: Whether to download bar files. True by default.
    :type download: bool

    :param extract: Whether to extract bar files. True by default.
    :type extract: bool

    :param signed: Whether to delete signed files. True by default.
    :type signed: bool

    :param compmethod: Compression method. Default is "7z", fallback "zip".
    :type compmethod: str

    :param gpg: Whether to use GnuPG verification. False by default.
    :type gpg: bool

    :param integrity: Whether to test downloaded files. True by default.
    :type integrity: bool

    :param altsw: Radio software release, if not the same as OS.
    :type altsw: str

    :param core: Whether to create a core/radio loader. Default is false.
    :type core: bool

    :param oldstyle: Whether to make old-style checksum files. Default is false.
    :type oldstyle: bool
    """
    if deleted and not compressed:
        deleted = False  # don't delete what we want to keep
    radioversion = scriptutils.return_radio_version(osversion, radioversion)
    softwareversion, swchecked = scriptutils.return_sw_checked(softwareversion, osversion)
    if altsw == "checkme":
        altsw, altchecked = scriptutils.return_radio_sw_checked(altsw, radioversion)
    localdir = utilities.dirhandler(localdir, os.getcwd())
    if hashed and hashdict is None:
        hashdict = hashutils.verifier_config_loader()
        hashutils.verifier_config_writer(hashdict)
    argutils.standard_preamble("archivist", osversion, softwareversion, radioversion, altsw)
    # Generate download URLs
    baseurl, alturl = scriptutils.get_baseurls(softwareversion, altsw)
    osurls, radiourls, cores = utilities.generate_urls(softwareversion, osversion, radioversion, True)
    osurls = cores if core else osurls
    for idx, url in enumerate(osurls):
        if "qc8960.factory_sfi" in url:
            vzwurl = url
            vzwindex = idx
            break
    if not networkutils.availability(vzwurl):
        osurls[vzwindex] = osurls[vzwindex].replace("qc8960.factory_sfi", "qc8960.verizon_sfi")
    osurls = list(set(osurls))  # pop duplicates
    if altsw:
        radiourls2 = [x.replace(baseurl, alturl) for x in radiourls]
        radiourls = radiourls2
        del radiourls2
    radiourls = list(set(radiourls))  # pop duplicates
    archivist_checksw(baseurl, softwareversion, swchecked)
    if altsw:
        scriptutils.check_radio_sw(alturl, altsw, altchecked)
    # Check availability of OS, radio
    scriptutils.check_os_bulk(osurls)
    radiourls, radioversion = scriptutils.check_radio_bulk(radiourls, radioversion)
    # Get 7z executable
    compmethod, szexe = scriptutils.get_sz_executable(compmethod)
    # Make dirs: bd_o, bd_r, ld_o, ld_r, zd_o, zd_r
    dirs = barutils.make_dirs(localdir, osversion, radioversion)
    osurls = scriptutils.bulk_avail(osurls)
    radiourls = scriptutils.bulk_avail(radiourls)
    sess = requests.Session()
    archivist_download(download, osurls, radiourls, localdir, sess, dirs)
    archivist_integritybars(integrity, osurls, radiourls, localdir)
    archivist_extractbars(extract, localdir)
    archivist_integritysigned(extract, localdir)
    archivist_movebars(dirs, localdir)
    archivist_generateloaders(osversion, radioversion, radios, localdir, altsw, core)
    archivist_integrityloaders(integrity, localdir)
    archivist_removesigned(signed, localdir)
    archivist_compressor(compressed, integrity, localdir, compmethod, szexe)
    archivist_moveloaders(dirs, localdir)
    archivist_gethashes(dirs, hashed, compressed, deleted, radios, osversion, radioversion, softwareversion, oldstyle)
    archivist_getoldhashes(dirs, hashed, compressed, deleted, radios, hashdict, oldstyle)
    archivist_gpg(gpg, dirs, compressed, deleted, radios)
    archivist_deleteuncompressed(dirs, deleted, radios)
    archivist_removeemptyfolders(localdir)
    print("\nFINISHED!")


if __name__ == "__main__":
    grab_args()
    decorators.enter_to_exit(False)


1			#!/usr/bin/env python3
2			"""Download bar files, create autoloaders."""
3
4			import os # filesystem read
5			import sys # load arguments
6
7			import requests # session
8			from bbarchivist import archiveutils # archive work
9			from bbarchivist import argutils # arguments
10			from bbarchivist import barutils # file/folder work
11			from bbarchivist import decorators # timer
12			from bbarchivist import hashutils # hashes, GPG
13			from bbarchivist import loadergen # cap, in Python
14			from bbarchivist import networkutils # download/lookup
15			from bbarchivist import scriptutils # script stuff
16			from bbarchivist import utilities # input validation
17
18			__author__ = "Thurask"
19			__license__ = "WTFPL v2"
20			__copyright__ = "2015-2019 Thurask"
21
22
23			@decorators.timer
24			def grab_args():
25			"""
26			Parse arguments from argparse/questionnaire.
27
28			Invoke :func:`archivist.archivist_main` with those arguments.
29			"""
30			if len(sys.argv) > 1:
31			parser = argutils.default_parser("bb-archivist", "Create autoloaders", ("folder", "osr"))
32			negategroup = parser.add_argument_group(
33			"negators",
34			"Disable program functionality")
35			negategroup.add_argument(
36			"-no",
37			"--no-download",
38			dest="download",
39			help="Don't download files",
40			action="store_false",
41			default=True)
42			negategroup.add_argument(
43			"-ni",
44			"--no-integrity",
45			dest="integrity",
46			help="Don't test bar files after download",
47			action="store_false",
48			default=True)
49			negategroup.add_argument(
50			"-nx",
51			"--no-extract",
52			dest="extract",
53			help="Don't extract bar files",
54			action="store_false",
55			default=True)
56			negategroup.add_argument(
57			"-nr",
58			"--no-radios",
59			dest="radloaders",
60			help="Don't make radio autoloaders",
61			action="store_false",
62			default=True)
63			negategroup.add_argument(
64			"-ns",
65			"--no-rmsigned",
66			dest="signed",
67			help="Don't remove signed files",
68			action="store_false",
69			default=True)
70			negategroup.add_argument(
71			"-nc",
72			"--no-compress",
73			dest="compress",
74			help="Don't compress loaders",
75			action="store_false",
76			default=True)
77			negategroup.add_argument(
78			"-nd",
79			"--no-delete",
80			dest="delete",
81			help="Don't delete uncompressed loaders",
82			action="store_false",
83			default=True)
84			negategroup.add_argument(
85			"-nv",
86			"--no-verify",
87			dest="verify",
88			help="Don't verify created loaders",
89			action="store_false",
90			default=True)
91			if not getattr(sys, 'frozen', False):
92			parser.add_argument(
93			"-g",
94			"--gpg",
95			dest="gpg",
96			help="Enable GPG signing. Set up GnuPG.",
97			action="store_true",
98			default=False)
99			parser.add_argument(
100			"-r",
101			"--radiosw",
102			dest="altsw",
103			metavar="SW",
104			help="Radio software version; use without software to guess",
105			nargs="?",
106			const="checkme",
107			default=None)
108			if not getattr(sys, 'frozen', False):
109			parser.add_argument(
110			"-m",
111			"--method",
112			dest="method",
113			metavar="METHOD",
114			help="Compression method",
115			nargs="?",
116			type=argutils.valid_method,
117			default=None)
118			parser.add_argument(
119			"-c",
120			"--core",
121			dest="core",
122			help="Make core/radio loaders",
123			default=False,
124			action="store_true")
125			parser.add_argument(
126			"-o",
127			"--old-style",
128			dest="oldstyle",
129			help="Make old-style checksum files",
130			default=False,
131			action="store_true")
132			parser.set_defaults(compmethod="7z")
133			args = parser.parse_args(sys.argv[1:])
134			args.folder = scriptutils.generate_workfolder(args.folder)
135			if getattr(sys, 'frozen', False):
136			args.gpg = False
137			hashdict = hashutils.verifier_config_loader(os.getcwd())
138			args.method = "7z"
139			else:
140			hashdict = hashutils.verifier_config_loader()
141			hashutils.verifier_config_writer(hashdict)
142			if args.method is None:
143			compmethod = archiveutils.compress_config_loader()
144			archiveutils.compress_config_writer(compmethod)
145			else:
146			compmethod = args.method
147			archivist_main(args.os, args.radio, args.swrelease,
148			os.path.abspath(args.folder), args.radloaders,
149			args.compress, args.delete, args.verify,
150			hashdict, args.download,
151			args.extract, args.signed, compmethod, args.gpg,
152			args.integrity, args.altsw, args.core, args.oldstyle)
153			else:
154			questionnaire()
155
156
157			def questionnaire():
158			"""
159			Questions to ask if no arguments given.
160			"""
161			localdir = os.getcwd()
162			osversion = input("OS VERSION (REQUIRED): ")
163			radioversion = input("RADIO VERSION (PRESS ENTER TO GUESS): ")
164			radioversion = None if not radioversion else radioversion
165			softwareversion = input("OS SOFTWARE RELEASE (PRESS ENTER TO GUESS): ")
166			softwareversion = None if not softwareversion else softwareversion
167			altcheck = utilities.i2b("USING ALTERNATE RADIO (Y/N)?: ")
168			altsw = scriptutils.check_altsw(altcheck)
169			radios = utilities.i2b("CREATE RADIO LOADERS (Y/N)?: ")
170			compressed = utilities.i2b("COMPRESS LOADERS (Y/N)?: ")
171			deleted = utilities.i2b("DELETE UNCOMPRESSED LOADERS (Y/N)?: ") if compressed else False
172			hashed = utilities.i2b("GENERATE HASHES (Y/N)?: ")
173			if getattr(sys, 'frozen', False):
174			hashdict = hashutils.verifier_config_loader(os.getcwd())
175			compmethod = "7z"
176			else:
177			hashdict = hashutils.verifier_config_loader()
178			hashutils.verifier_config_writer(hashdict)
179			compmethod = archiveutils.compress_config_loader()
180			print(" ")
181			archivist_main(osversion, radioversion, softwareversion,
182			localdir, radios, compressed, deleted, hashed,
183			hashdict, download=True, extract=True, signed=True,
184			compmethod=compmethod, gpg=False, integrity=True,
185			altsw=altsw, core=False, oldstyle=False)
186
187
188			def archivist_checksw(baseurl, softwareversion, swchecked):
189			"""
190			Check availability of software releases.
191
192			:param baseurl: Base URL for download links.
193			:type baseurl: str
194
195			:param softwareversion: Software release, 10.x.y.zzzz. Can be guessed.
196			:type softwareversion: str
197
198			:param swchecked: If we checked the sw release already.
199			:type swchecked: bool
200			"""
201			scriptutils.check_sw(baseurl, softwareversion, swchecked)
202
203
204			def archivist_download(download, osurls, radiourls, localdir, session, dirs):
205			"""
206			Download function.
207
208			:param download: Whether to download bar files. True by default.
209			:type download: bool
210
211			:param osurls: OS file list.
212			:type osurls: list(str)
213
214			:param radiourls: Radio file list.
215			:type radiourls: list(str)
216
217			:param localdir: Working directory. Local by default.
218			:type localdir: str
219
220			:param session: Requests session object, default is created on the fly.
221			:type session: requests.Session()
222
223			:param dirs: List of generated bar/loader/zip directories.
224			:type dirs: list(str)
225			"""
226			osfiles = scriptutils.comp_joiner(localdir, dirs[0], osurls)
227			radiofiles = scriptutils.comp_joiner(localdir, dirs[1], radiourls)
228			if download:
229			print("BEGIN DOWNLOADING...")
230			networkutils.download_bootstrap(radiourls + osurls, localdir, 3, session)
231			print("ALL FILES DOWNLOADED")
232			elif all(os.path.exists(x) for x in osfiles+radiofiles):
233			print("USING CACHED OS/RADIO FILES...")
234			barutils.replace_bars_bulk(os.path.abspath(localdir), osfiles+radiofiles)
235
236
237			def archivist_integritybars(integrity, osurls, radiourls, localdir):
238			"""
239			Check integrity of bar files, redownload if necessary.
240
241			:param integrity: Whether to test downloaded files. True by default.
242			:type integrity: bool
243
244			:param osurls: OS file list.
245			:type osurls: list(str)
246
247			:param radiourls: Radio file list.
248			:type radiourls: list(str)
249			"""
250			if integrity:
251			urllist = osurls + radiourls
252			scriptutils.test_bar_files(localdir, urllist)
253
254
255			def archivist_extractbars(extract, localdir):
256			"""
257			Extract signed files from bar files.
258
259			:param extract: Whether to extract bar files. True by default.
260			:type extract: bool
261
262			:param localdir: Working directory. Local by default.
263			:type localdir: str
264			"""
265			if extract:
266			print("EXTRACTING...")
267			barutils.extract_bars(localdir)
268
269
270			def archivist_integritysigned(integrity, localdir):
271			"""
272			Check integrity of signed files.
273
274			:param integrity: Whether to test downloaded files. True by default.
275			:type integrity: bool
276
277			:param localdir: Working directory. Local by default.
278			:type localdir: str
279			"""
280			if integrity:
281			scriptutils.test_signed_files(localdir)
282
283
284			def archivist_movebars(dirs, localdir):
285			"""
286			Move bar files.
287
288			:param dirs: List of OS/radio bar/loader/zipped folders.
289			:type dirs: list(str)
290
291			:param localdir: Working directory. Local by default.
292			:type localdir: str
293			"""
294			print("MOVING BAR FILES...")
295			barutils.move_bars(localdir, dirs[0], dirs[1])
296
297
298			def archivist_generateloaders(osversion, radioversion, radios, localdir, altsw, core):
299			"""
300			Generate loaders.
301
302			:param osversion: OS version, 10.x.y.zzzz. Required.
303			:type osversion: str
304
305			:param radioversion: Radio version, 10.x.y.zzzz. Can be guessed.
306			:type radioversion: str
307
308			:param radios: Whether to create radio autoloaders. True by default.
309			:type radios: bool
310
311			:param localdir: Working directory. Local by default.
312			:type localdir: str
313
314			:param altsw: Radio software release, if not the same as OS.
315			:type altsw: str
316
317			:param core: Whether to create a core/radio loader. Default is false.
318			:type core: bool
319			"""
320			print("GENERATING LOADERS...")
321			altradio = altsw is not None
322			loadergen.generate_loaders(osversion, radioversion, radios, localdir, altradio, core)
323
324
325			def archivist_integrityloaders(integrity, localdir):
326			"""
327			Check integrity of build loaders.
328
329			:param integrity: Whether to test downloaded files. True by default.
330			:type integrity: bool
331
332			:param localdir: Working directory. Local by default.
333			:type localdir: str
334			"""
335			if integrity:
336			scriptutils.test_loader_files(localdir)
337
338
339			def archivist_removesigned(signed, localdir):
340			"""
341			Remove signed files.
342
343			:param signed: Whether to delete signed files. True by default.
344			:type signed: bool
345
346			:param localdir: Working directory. Local by default.
347			:type localdir: str
348			"""
349			if signed:
350			print("REMOVING SIGNED FILES...")
351			barutils.remove_signed_files(localdir)
352
353
354			def archivist_compressor(compressed, integrity, localdir, compmethod, szexe):
355			"""
356			Compress and optionally verify loaders.
357
358			:param compressed: Whether to compress files. True by default.
359			:type compressed: bool
360
361			:param integrity: Whether to test downloaded files. True by default.
362			:type integrity: bool
363
364			:param localdir: Working directory. Local by default.
365			:type localdir: str
366
367			:param compmethod: Compression method. Default is "7z", fallback "zip".
368			:type compmethod: str
369
370			:param szexe: Path to 7z executable.
371			:type szexe: str
372			"""
373			if compressed:
374			print("COMPRESSING...")
375			archiveutils.compress(localdir, compmethod, szexe, True)
376			if integrity:
377			print("TESTING ARCHIVES...")
378			archiveutils.verify(localdir, compmethod, szexe, True)
379
380
381			def archivist_moveloaders(dirs, localdir):
382			"""
383			Move loaders.
384
385			:param dirs: List of OS/radio bar/loader/zipped folders.
386			:type dirs: list(str)
387
388			:param localdir: Working directory. Local by default.
389			:type localdir: str
390			"""
391			print("MOVING LOADERS...")
392			barutils.move_loaders(localdir, dirs[2], dirs[3], dirs[4], dirs[5])
393
394
395			def archivist_gethashes(dirs, hashed, compressed, deleted, radios, osversion, radioversion, softwareversion, oldstyle):
396			"""
397			Make new-style info files.
398
399			:param dirs: List of OS/radio bar/loader/zipped folders.
400			:type dirs: list(str)
401
402			:param hashed: Whether to hash files. True by default.
403			:type hashed: bool
404
405			:param compressed: Whether to compress files. True by default.
406			:type compressed: bool
407
408			:param deleted: Whether to delete uncompressed files. True by default.
409			:type deleted: bool
410
411			:param radios: Whether to create radio autoloaders. True by default.
412			:type radios: bool
413
414			:param osversion: OS version, 10.x.y.zzzz. Required.
415			:type osversion: str
416
417			:param radioversion: Radio version, 10.x.y.zzzz. Can be guessed.
418			:type radioversion: str
419
420			:param softwareversion: Software release, 10.x.y.zzzz. Can be guessed.
421			:type softwareversion: str
422
423			:param oldstyle: Whether to make old-style checksum files. Default is false.
424			:type oldstyle: bool
425			"""
426			if hashed and not oldstyle:
427			scriptutils.bulk_info(dirs, osversion, compressed, deleted, radios, radioversion, softwareversion)
428
429			def archivist_getoldhashes(dirs, hashed, compressed, deleted, radios, hashdict, oldstyle):
430			"""
431			Make old-style checksum files.
432
433			:param dirs: List of OS/radio bar/loader/zipped folders.
434			:type dirs: list(str)
435
436			:param hashed: Whether to hash files. True by default.
437			:type hashed: bool
438
439			:param compressed: Whether to compress files. True by default.
440			:type compressed: bool
441
442			:param deleted: Whether to delete uncompressed files. True by default.
443			:type deleted: bool
444
445			:param radios: Whether to create radio autoloaders. True by default.
446			:type radios: bool
447
448			:param hashdict: Dictionary of hash rules, in ~\bbarchivist.ini.
449			:type hashdict: dict({str: bool})
450
451			:param oldstyle: Whether to make old-style checksum files. Default is false.
452			:type oldstyle: bool
453			"""
454			if hashed and oldstyle:
455			scriptutils.bulk_hash(dirs, compressed, deleted, radios, hashdict)
456
457
458			def archivist_gpg(gpg, dirs, compressed, deleted, radios):
459			"""
460			GPG-sign everything.
461
462			:param gpg: Whether to use GnuPG verification. False by default.
463			:type gpg: bool
464
465			:param dirs: List of OS/radio bar/loader/zipped folders.
466			:type dirs: list(str)
467
468			:param compressed: Whether to compress files. True by default.
469			:type compressed: bool
470
471			:param deleted: Whether to delete uncompressed files. True by default.
472			:type deleted: bool
473
474			:param radios: Whether to create radio autoloaders. True by default.
475			:type radios: bool
476			"""
477			if gpg:
478			scriptutils.bulk_verify(dirs, compressed, deleted, radios)
479
480
481			def archivist_deleteuncompressed(dirs, deleted, radios):
482			"""
483			Delete uncompressed loaders.
484
485			:param dirs: List of OS/radio bar/loader/zipped folders.
486			:type dirs: list(str)
487
488			:param deleted: Whether to delete uncompressed files. True by default.
489			:type deleted: bool
490
491			:param radios: Whether to create radio autoloaders. True by default.
492			:type radios: bool
493			"""
494			if deleted:
495			print("DELETING UNCOMPRESSED LOADERS...")
496			barutils.remove_unpacked_loaders(dirs[2], dirs[3], radios)
497
498
499			def archivist_removeemptyfolders(localdir):
500			"""
501			Delete empty folders.
502
503			:param localdir: Working directory. Local by default.
504			:type localdir: str
505			"""
506			print("REMOVING EMPTY FOLDERS...")
507			barutils.remove_empty_folders(localdir)
508
509
510			def archivist_main(osversion, radioversion=None, softwareversion=None,
511			localdir=None, radios=True, compressed=True, deleted=True,
512			hashed=True, hashdict=None, download=True,
513			extract=True, signed=True, compmethod="7z",
514			gpg=False, integrity=True, altsw=None,
515			core=False, oldstyle=False):
516			"""
517			Wrap around multi-autoloader creation code.
518			Some combination of creating, downloading, hashing,
519			compressing and moving autoloaders.
520
521			:param osversion: OS version, 10.x.y.zzzz. Required.
522			:type osversion: str
523
524			:param radioversion: Radio version, 10.x.y.zzzz. Can be guessed.
525			:type radioversion: str
526
527			:param softwareversion: Software release, 10.x.y.zzzz. Can be guessed.
528			:type softwareversion: str
529
530			:param localdir: Working directory. Local by default.
531			:type localdir: str
532
533			:param radios: Whether to create radio autoloaders. True by default.
534			:type radios: bool
535
536			:param compressed: Whether to compress files. True by default.
537			:type compressed: bool
538
539			:param deleted: Whether to delete uncompressed files. True by default.
540			:type deleted: bool
541
542			:param hashed: Whether to hash files. True by default.
543			:type hashed: bool
544
545			:param hashdict: Dictionary of hash rules, in ~\bbarchivist.ini.
546			:type hashdict: dict({str: bool})
547
548			:param download: Whether to download bar files. True by default.
549			:type download: bool
550
551			:param extract: Whether to extract bar files. True by default.
552			:type extract: bool
553
554			:param signed: Whether to delete signed files. True by default.
555			:type signed: bool
556
557			:param compmethod: Compression method. Default is "7z", fallback "zip".
558			:type compmethod: str
559
560			:param gpg: Whether to use GnuPG verification. False by default.
561			:type gpg: bool
562
563			:param integrity: Whether to test downloaded files. True by default.
564			:type integrity: bool
565
566			:param altsw: Radio software release, if not the same as OS.
567			:type altsw: str
568
569			:param core: Whether to create a core/radio loader. Default is false.
570			:type core: bool
571
572			:param oldstyle: Whether to make old-style checksum files. Default is false.
573			:type oldstyle: bool
574			"""
575			if deleted and not compressed:
576			deleted = False # don't delete what we want to keep
577			radioversion = scriptutils.return_radio_version(osversion, radioversion)
578			softwareversion, swchecked = scriptutils.return_sw_checked(softwareversion, osversion)
579			if altsw == "checkme":
580			altsw, altchecked = scriptutils.return_radio_sw_checked(altsw, radioversion)
581			localdir = utilities.dirhandler(localdir, os.getcwd())
582			if hashed and hashdict is None:
583			hashdict = hashutils.verifier_config_loader()
584			hashutils.verifier_config_writer(hashdict)
585			argutils.standard_preamble("archivist", osversion, softwareversion, radioversion, altsw)
586			# Generate download URLs
587			baseurl, alturl = scriptutils.get_baseurls(softwareversion, altsw)
588			osurls, radiourls, cores = utilities.generate_urls(softwareversion, osversion, radioversion, True)
589			osurls = cores if core else osurls
590			for idx, url in enumerate(osurls):
591			if "qc8960.factory_sfi" in url:
592			vzwurl = url
593			vzwindex = idx
594			break
595			if not networkutils.availability(vzwurl):
596			osurls[vzwindex] = osurls[vzwindex].replace("qc8960.factory_sfi", "qc8960.verizon_sfi")
597			osurls = list(set(osurls)) # pop duplicates
598			if altsw:
599			radiourls2 = [x.replace(baseurl, alturl) for x in radiourls]
600			radiourls = radiourls2
601			del radiourls2
602			radiourls = list(set(radiourls)) # pop duplicates
603			archivist_checksw(baseurl, softwareversion, swchecked)
604			if altsw:
605			scriptutils.check_radio_sw(alturl, altsw, altchecked)
606			# Check availability of OS, radio
607			scriptutils.check_os_bulk(osurls)
608			radiourls, radioversion = scriptutils.check_radio_bulk(radiourls, radioversion)
609			# Get 7z executable
610			compmethod, szexe = scriptutils.get_sz_executable(compmethod)
611			# Make dirs: bd_o, bd_r, ld_o, ld_r, zd_o, zd_r
612			dirs = barutils.make_dirs(localdir, osversion, radioversion)
613			osurls = scriptutils.bulk_avail(osurls)
614			radiourls = scriptutils.bulk_avail(radiourls)
615			sess = requests.Session()
616			archivist_download(download, osurls, radiourls, localdir, sess, dirs)
617			archivist_integritybars(integrity, osurls, radiourls, localdir)
618			archivist_extractbars(extract, localdir)
619			archivist_integritysigned(extract, localdir)
620			archivist_movebars(dirs, localdir)
621			archivist_generateloaders(osversion, radioversion, radios, localdir, altsw, core)
622			archivist_integrityloaders(integrity, localdir)
623			archivist_removesigned(signed, localdir)
624			archivist_compressor(compressed, integrity, localdir, compmethod, szexe)
625			archivist_moveloaders(dirs, localdir)
626			archivist_gethashes(dirs, hashed, compressed, deleted, radios, osversion, radioversion, softwareversion, oldstyle)
627			archivist_getoldhashes(dirs, hashed, compressed, deleted, radios, hashdict, oldstyle)
628			archivist_gpg(gpg, dirs, compressed, deleted, radios)
629			archivist_deleteuncompressed(dirs, deleted, radios)
630			archivist_removeemptyfolders(localdir)
631			print("\nFINISHED!")
632
633
634			if __name__ == "__main__":
635			grab_args()
636			decorators.enter_to_exit(False)
637

thurask / bbarchivist

bbarchivist.scripts.archivist.archivist_main() D last analyzed 2019-11-17 11:32 UTC

Complexity

Size

Duplication

Importance

How to fix Long Method Complexity Many Parameters

Long Method

Complexity

Many Parameters

Duplication Side-by-Side

Filter issues like

bbarchivist.scripts.archivist.archivist_main() D
last analyzed 2019-11-17 11:32 UTC