NMRShiftDB2Converter

Complexity

Total Complexity

41

Size/Duplication

Total Lines	158
Duplicated Lines	0 %

Importance

Changes	1
Bugs	0	Features	1

20 Methods

Rating	Name	Duplication	Size	Complexity
B	log_dists()	0	13	6
A	is_spectrum()	0	2	2
A	__init__()	0	22	1
A	index_pair()	0	2	1
A	is_duplicate()	0	5	2
A	plot_duplicates()	0	4	1
B	combine_duplicates()	0	13	7
A	squash_duplicates()	0	11	4
A	log_duplicates()	0	10	4
A	log_message()	0	2	2
B	process_spectra()	0	14	6
B	aggregate_dicts()	0	7	5
A	squash()	0	5	3
A	parse_data()	0	12	1
B	spectrum_dict()	0	8	5
A	to_frame()	0	6	1
A	n_shifts()	0	2	2
C	get_spectra()	0	30	7
A	n_spect()	0	2	1
A	extract_duplicates()	0	11	3

#! /usr/bin/env python

This module should have a docstring.

#
# Copyright (C) 2016 Rich Lewis <rl403@cam.ac.uk>
# License: 3-clause BSD

import os
import logging
import itertools
from collections import defaultdict

import pandas as pd

The import pandas could not be resolved.

import numpy as np

The import numpy could not be resolved.

from sklearn import metrics

The import sklearn could not be resolved.

from .base import Converter

Unable to import 'base' (invalid syntax (<string>, line 46))

from ... import io
from ... import utils

LOGGER = logging.getLogger(__file__)

class NMRShiftDB2Converter(Converter):

This class should have a docstring.

    def __init__(self, directory, output_directory, output_filename='nmrshiftdb2.h5'):

        output_path = os.path.join(output_directory, output_filename)
        input_path = os.path.join(directory, 'nmrshiftdb2.sdf')
        data = self.parse_data(input_path)

        ys = self.get_spectra(data)

The name ys does not conform to the variable naming conventions ([a-z_][a-z0-9_]{2,30}$).

        ys = self.process_spectra(ys)

The name ys does not conform to the variable naming conventions ([a-z_][a-z0-9_]{2,30}$).

        ys = self.combine_duplicates(ys)

The name ys does not conform to the variable naming conventions ([a-z_][a-z0-9_]{2,30}$).

        self.log_dists(ys)
        self.log_duplicates(ys)
        ys = self.squash_duplicates(ys)

The name ys does not conform to the variable naming conventions ([a-z_][a-z0-9_]{2,30}$).

        c13s = self.to_frame(ys.loc[ys['13c'].notnull(), '13c'])
        data = data[['structure']].join(c13s, how='right')

        data = self.standardize(data)

The Instance of NMRShiftDB2Converter does not seem to have a member named standardize.

        data = self.filter(data)

The Instance of NMRShiftDB2Converter does not seem to have a member named filter.

        data = self.optimize(data)

The Instance of NMRShiftDB2Converter does not seem to have a member named optimize.

        ms, y = data.structure, data.drop('structure', axis=1)

The name ms does not conform to the variable naming conventions ([a-z_][a-z0-9_]{2,30}$).

The name y does not conform to the variable naming conventions ([a-z_][a-z0-9_]{2,30}$).

        self.run(ms, y, output_path=output_path)

The Instance of NMRShiftDB2Converter does not seem to have a member named run.

    def parse_data(self, filepath):

This method could be written as a function/class method.

        """ Reads the raw datafile. """

        LOGGER.info('Reading file: %s', filepath)
        data = io.read_sdf(filepath, removeHs=False, warn_bad_mol=False)
        data.index = data['nmrshiftdb2 ID'].astype(int)
        data.index.name = 'nmrshiftdb2_id'
        data.columns = data.columns.to_series().apply(utils.free_to_snail)
        data = data.sort_index()
        LOGGER.info('Read %s molecules.', len(data))
        return data

    def get_spectra(self, data):

This method could be written as a function/class method.

        """ Retrieves spectra from raw data. """

        LOGGER.info('Retrieving spectra from raw data...')
        isotopes = [
            '1h',
            '11b',
            '13c',
            '15n',
            '17o',
            '19f',
            '29si',
            '31p',
            '33s',
            '73ge',
            '195pt'
        ]

        def is_spectrum(col_name, ele='c'):

This function should have a docstring.

The argument ele seems to be unused.

            return any(isotope in col_name for isotope in isotopes)

        spectrum_cols = [c for c in data if is_spectrum(c)]
        data = data[spectrum_cols]

        def index_pair(s):

The name s does not conform to the argument naming conventions ([a-z_][a-z0-9_]{2,30}$).

This function should have a docstring.

            return s[0], int(s[1])

        data.columns = pd.MultiIndex.from_tuples([index_pair(i.split('_')[1:]) for i in data.columns])

This line is too long as per the coding-style (102/100).

        return data

    def process_spectra(self, data):

This method could be written as a function/class method.

        """ Turn the string representations found in sdf file into a dictionary. """

        def spectrum_dict(spectrum_string):

This function should have a docstring.

            if not isinstance(spectrum_string, str):
                return np.nan # no spectra are still nan
            if spectrum_string == '':
                return np.nan # empty spectra are nan
            sigs = spectrum_string.strip().strip('|').strip().split('|') # extract signals
            sig_tup = [tuple(s.split(';')) for s in sigs] # take tuples as (signal, coupling, atom)
            return {int(s[2]): float(s[0]) for s in sig_tup} # make spectrum a dictionary of atom to signal

This line is too long as per the coding-style (107/100).

        return data.applymap(spectrum_dict)

    def combine_duplicates(self, data):

This method could be written as a function/class method.

        """ Collect duplicate spectra into one dictionary. All shifts are collected into lists. """

        def aggregate_dicts(ds):

The name ds does not conform to the argument naming conventions ([a-z_][a-z0-9_]{2,30}$).

This function should have a docstring.

            res = defaultdict(list)
            for d in ds:

The name d does not conform to the variable naming conventions ([a-z_][a-z0-9_]{2,30}$).

                if not isinstance(d, dict): continue

More than one statement on a single line

                for k, v in d.items():

The name v does not conform to the variable naming conventions ([a-z_][a-z0-9_]{2,30}$).

                    res[k].append(v)
            return dict(res) if len(res) else np.nan

        return data.groupby(level=0, axis=1).apply(lambda s: s.apply(aggregate_dicts, axis=1))

    def squash_duplicates(self, data):

This method could be written as a function/class method.

        """ Take the mean of all the duplicates.  This is where we could do a bit more checking. """

        def squash(d):

The name d does not conform to the argument naming conventions ([a-z_][a-z0-9_]{2,30}$).

This function should have a docstring.

            if not isinstance(d, dict):
                return np.nan
            else:
                return {k: np.mean(v) for k, v in d.items()}

        return data.applymap(squash)

    def to_frame(self, data):

This method could be written as a function/class method.

        """ Convert a series of dictionaries to a dataframe. """
        res = pd.DataFrame(data.tolist(), index=data.index)
        res.columns.name = 'atom_idx'
        return res

    def extract_duplicates(self, data, kind='13c'):

This method could be written as a function/class method.

        """ Get all 13c duplicates.  """

        def is_duplicate(ele):

This function should have a docstring.

            if not isinstance(ele, dict):
                return False
            else:
                return len(list(ele.values())[0]) > 1

        return data.loc[data[kind].apply(is_duplicate), kind]

    def log_dists(self, data):

This method should have a docstring.

This method could be written as a function/class method.

        def n_spect(ele):

This function should have a docstring.

            return isinstance(ele, dict)

        def n_shifts(ele):

This function should have a docstring.

            return len(ele) if isinstance(ele, dict) else 0

        def log_message(func):

This function should have a docstring.

            return '  '.join('{k}: {v}'.format(k=k, v=v) for k, v in data.applymap(func).sum().to_dict().items())

This line is too long as per the coding-style (113/100).

        LOGGER.info('Number of spectra: %s', log_message(n_spect))
        LOGGER.info('Extracted shifts: %s', log_message(n_shifts))

    def log_duplicates(self, data):

This method should have a docstring.

        for kind in '1h', '13c':
            dups = self.extract_duplicates(data, kind)
            LOGGER.info('Number of duplicate %s spectra: %s', kind, len(dups))
            res = pd.DataFrame(sum((list(itertools.combinations(l, 2)) for s in dups for k, l in s.items()), []))

This line is too long as per the coding-style (113/100).

            LOGGER.info('Number of duplicate %s pairs: %f', kind, len(res))
            LOGGER.info('MAE for duplicate %s: %.4f', kind, metrics.mean_absolute_error(res[0], res[1]))

This line is too long as per the coding-style (104/100).

            LOGGER.info('MSE for duplicate %s: %.4f', kind, metrics.mean_squared_error(res[0], res[1]))

This line is too long as per the coding-style (103/100).

            LOGGER.info('r2 for duplicate %s: %.4f', kind, metrics.r2_score(res[0], res[1]))

    def plot_duplicates(self, data):

        """ Plot the duplicates """
        pass



if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)
    LOGGER.info('Converting NMRShiftDB2 Dataset...')
    NMRShiftDB2Converter.convert()

Final newline missing

The Class NMRShiftDB2Converter does not seem to have a member named convert.