sklearn_numeric_clustering: train_test

annotate train_test_eval.py @ 36:836ba896e2be draft

"planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit d6333e7294e67be5968a41f404b66699cad4ae53"

author	bgruening
date	Thu, 07 Nov 2019 05:13:53 -0500
parents	fbd849199283
children	80bb86a40de6

rev	line source
32 a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	1 import argparse
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	2 import joblib
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	3 import json
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	4 import numpy as np
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	5 import pandas as pd
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	6 import pickle
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	7 import warnings
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	8 from itertools import chain
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	9 from scipy.io import mmread
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	10 from sklearn.base import clone
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	11 from sklearn import (cluster, compose, decomposition, ensemble,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	12 feature_extraction, feature_selection,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	13 gaussian_process, kernel_approximation, metrics,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	14 model_selection, naive_bayes, neighbors,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	15 pipeline, preprocessing, svm, linear_model,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	16 tree, discriminant_analysis)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	17 from sklearn.exceptions import FitFailedWarning
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	18 from sklearn.metrics.scorer import _check_multimetric_scoring
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	19 from sklearn.model_selection._validation import _score, cross_validate
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	20 from sklearn.model_selection import _search, _validation
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	21 from sklearn.utils import indexable, safe_indexing
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	22
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	23 from galaxy_ml.model_validations import train_test_split
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	24 from galaxy_ml.utils import (SafeEval, get_scoring, load_model,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	25 read_columns, try_get_attr, get_module)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	26
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	27
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	28 _fit_and_score = try_get_attr('galaxy_ml.model_validations', '_fit_and_score')
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	29 setattr(_search, '_fit_and_score', _fit_and_score)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	30 setattr(_validation, '_fit_and_score', _fit_and_score)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	31
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	32 N_JOBS = int(__import__('os').environ.get('GALAXY_SLOTS', 1))
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	33 CACHE_DIR = './cached'
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	34 NON_SEARCHABLE = ('n_jobs', 'pre_dispatch', 'memory', '_path',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	35 'nthread', 'callbacks')
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	36 ALLOWED_CALLBACKS = ('EarlyStopping', 'TerminateOnNaN', 'ReduceLROnPlateau',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	37 'CSVLogger', 'None')
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	38
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	39
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	40 def _eval_swap_params(params_builder):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	41 swap_params = {}
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	42
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	43 for p in params_builder['param_set']:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	44 swap_value = p['sp_value'].strip()
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	45 if swap_value == '':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	46 continue
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	47
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	48 param_name = p['sp_name']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	49 if param_name.lower().endswith(NON_SEARCHABLE):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	50 warnings.warn("Warning: `%s` is not eligible for search and was "
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	51 "omitted!" % param_name)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	52 continue
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	53
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	54 if not swap_value.startswith(':'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	55 safe_eval = SafeEval(load_scipy=True, load_numpy=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	56 ev = safe_eval(swap_value)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	57 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	58 # Have `:` before search list, asks for estimator evaluatio
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	59 safe_eval_es = SafeEval(load_estimators=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	60 swap_value = swap_value[1:].strip()
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	61 # TODO maybe add regular express check
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	62 ev = safe_eval_es(swap_value)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	63
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	64 swap_params[param_name] = ev
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	65
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	66 return swap_params
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	67
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	68
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	69 def train_test_split_none(arrays, *kwargs):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	70 """extend train_test_split to take None arrays
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	71 and support split by group names.
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	72 """
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	73 nones = []
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	74 new_arrays = []
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	75 for idx, arr in enumerate(arrays):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	76 if arr is None:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	77 nones.append(idx)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	78 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	79 new_arrays.append(arr)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	80
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	81 if kwargs['shuffle'] == 'None':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	82 kwargs['shuffle'] = None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	83
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	84 group_names = kwargs.pop('group_names', None)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	85
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	86 if group_names is not None and group_names.strip():
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	87 group_names = [name.strip() for name in
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	88 group_names.split(',')]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	89 new_arrays = indexable(*new_arrays)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	90 groups = kwargs['labels']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	91 n_samples = new_arrays[0].shape[0]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	92 index_arr = np.arange(n_samples)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	93 test = index_arr[np.isin(groups, group_names)]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	94 train = index_arr[~np.isin(groups, group_names)]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	95 rval = list(chain.from_iterable(
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	96 (safe_indexing(a, train),
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	97 safe_indexing(a, test)) for a in new_arrays))
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	98 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	99 rval = train_test_split(new_arrays, *kwargs)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	100
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	101 for pos in nones:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	102 rval[pos * 2: 2] = [None, None]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	103
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	104 return rval
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	105
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	106
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	107 def main(inputs, infile_estimator, infile1, infile2,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	108 outfile_result, outfile_object=None,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	109 outfile_weights=None, groups=None,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	110 ref_seq=None, intervals=None, targets=None,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	111 fasta_path=None):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	112 """
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	113 Parameter
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	114 ---------
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	115 inputs : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	116 File path to galaxy tool parameter
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	117
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	118 infile_estimator : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	119 File path to estimator
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	120
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	121 infile1 : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	122 File path to dataset containing features
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	123
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	124 infile2 : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	125 File path to dataset containing target values
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	126
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	127 outfile_result : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	128 File path to save the results, either cv_results or test result
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	129
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	130 outfile_object : str, optional
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	131 File path to save searchCV object
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	132
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	133 outfile_weights : str, optional
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	134 File path to save deep learning model weights
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	135
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	136 groups : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	137 File path to dataset containing groups labels
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	138
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	139 ref_seq : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	140 File path to dataset containing genome sequence file
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	141
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	142 intervals : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	143 File path to dataset containing interval file
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	144
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	145 targets : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	146 File path to dataset compressed target bed file
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	147
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	148 fasta_path : str
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	149 File path to dataset containing fasta file
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	150 """
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	151 warnings.simplefilter('ignore')
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	152
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	153 with open(inputs, 'r') as param_handler:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	154 params = json.load(param_handler)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	155
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	156 # load estimator
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	157 with open(infile_estimator, 'rb') as estimator_handler:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	158 estimator = load_model(estimator_handler)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	159
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	160 # swap hyperparameter
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	161 swapping = params['experiment_schemes']['hyperparams_swapping']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	162 swap_params = _eval_swap_params(swapping)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	163 estimator.set_params(**swap_params)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	164
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	165 estimator_params = estimator.get_params()
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	166
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	167 # store read dataframe object
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	168 loaded_df = {}
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	169
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	170 input_type = params['input_options']['selected_input']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	171 # tabular input
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	172 if input_type == 'tabular':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	173 header = 'infer' if params['input_options']['header1'] else None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	174 column_option = (params['input_options']['column_selector_options_1']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	175 ['selected_column_selector_option'])
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	176 if column_option in ['by_index_number', 'all_but_by_index_number',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	177 'by_header_name', 'all_but_by_header_name']:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	178 c = params['input_options']['column_selector_options_1']['col1']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	179 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	180 c = None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	181
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	182 df_key = infile1 + repr(header)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	183 df = pd.read_csv(infile1, sep='\t', header=header,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	184 parse_dates=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	185 loaded_df[df_key] = df
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	186
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	187 X = read_columns(df, c=c, c_option=column_option).astype(float)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	188 # sparse input
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	189 elif input_type == 'sparse':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	190 X = mmread(open(infile1, 'r'))
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	191
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	192 # fasta_file input
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	193 elif input_type == 'seq_fasta':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	194 pyfaidx = get_module('pyfaidx')
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	195 sequences = pyfaidx.Fasta(fasta_path)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	196 n_seqs = len(sequences.keys())
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	197 X = np.arange(n_seqs)[:, np.newaxis]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	198 for param in estimator_params.keys():
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	199 if param.endswith('fasta_path'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	200 estimator.set_params(
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	201 **{param: fasta_path})
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	202 break
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	203 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	204 raise ValueError(
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	205 "The selected estimator doesn't support "
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	206 "fasta file input! Please consider using "
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	207 "KerasGBatchClassifier with "
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	208 "FastaDNABatchGenerator/FastaProteinBatchGenerator "
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	209 "or having GenomeOneHotEncoder/ProteinOneHotEncoder "
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	210 "in pipeline!")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	211
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	212 elif input_type == 'refseq_and_interval':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	213 path_params = {
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	214 'data_batch_generator__ref_genome_path': ref_seq,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	215 'data_batch_generator__intervals_path': intervals,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	216 'data_batch_generator__target_path': targets
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	217 }
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	218 estimator.set_params(**path_params)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	219 n_intervals = sum(1 for line in open(intervals))
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	220 X = np.arange(n_intervals)[:, np.newaxis]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	221
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	222 # Get target y
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	223 header = 'infer' if params['input_options']['header2'] else None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	224 column_option = (params['input_options']['column_selector_options_2']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	225 ['selected_column_selector_option2'])
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	226 if column_option in ['by_index_number', 'all_but_by_index_number',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	227 'by_header_name', 'all_but_by_header_name']:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	228 c = params['input_options']['column_selector_options_2']['col2']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	229 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	230 c = None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	231
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	232 df_key = infile2 + repr(header)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	233 if df_key in loaded_df:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	234 infile2 = loaded_df[df_key]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	235 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	236 infile2 = pd.read_csv(infile2, sep='\t',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	237 header=header, parse_dates=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	238 loaded_df[df_key] = infile2
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	239
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	240 y = read_columns(
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	241 infile2,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	242 c=c,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	243 c_option=column_option,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	244 sep='\t',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	245 header=header,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	246 parse_dates=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	247 if len(y.shape) == 2 and y.shape[1] == 1:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	248 y = y.ravel()
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	249 if input_type == 'refseq_and_interval':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	250 estimator.set_params(
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	251 data_batch_generator__features=y.ravel().tolist())
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	252 y = None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	253 # end y
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	254
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	255 # load groups
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	256 if groups:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	257 groups_selector = (params['experiment_schemes']['test_split']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	258 ['split_algos']).pop('groups_selector')
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	259
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	260 header = 'infer' if groups_selector['header_g'] else None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	261 column_option = \
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	262 (groups_selector['column_selector_options_g']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	263 ['selected_column_selector_option_g'])
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	264 if column_option in ['by_index_number', 'all_but_by_index_number',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	265 'by_header_name', 'all_but_by_header_name']:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	266 c = groups_selector['column_selector_options_g']['col_g']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	267 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	268 c = None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	269
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	270 df_key = groups + repr(header)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	271 if df_key in loaded_df:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	272 groups = loaded_df[df_key]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	273
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	274 groups = read_columns(
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	275 groups,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	276 c=c,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	277 c_option=column_option,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	278 sep='\t',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	279 header=header,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	280 parse_dates=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	281 groups = groups.ravel()
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	282
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	283 # del loaded_df
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	284 del loaded_df
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	285
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	286 # handle memory
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	287 memory = joblib.Memory(location=CACHE_DIR, verbose=0)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	288 # cache iraps_core fits could increase search speed significantly
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	289 if estimator.__class__.__name__ == 'IRAPSClassifier':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	290 estimator.set_params(memory=memory)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	291 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	292 # For iraps buried in pipeline
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	293 new_params = {}
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	294 for p, v in estimator_params.items():
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	295 if p.endswith('memory'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	296 # for case of `__irapsclassifier__memory`
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	297 if len(p) > 8 and p[:-8].endswith('irapsclassifier'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	298 # cache iraps_core fits could increase search
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	299 # speed significantly
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	300 new_params[p] = memory
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	301 # security reason, we don't want memory being
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	302 # modified unexpectedly
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	303 elif v:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	304 new_params[p] = None
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	305 # handle n_jobs
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	306 elif p.endswith('n_jobs'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	307 # For now, 1 CPU is suggested for iprasclassifier
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	308 if len(p) > 8 and p[:-8].endswith('irapsclassifier'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	309 new_params[p] = 1
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	310 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	311 new_params[p] = N_JOBS
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	312 # for security reason, types of callback are limited
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	313 elif p.endswith('callbacks'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	314 for cb in v:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	315 cb_type = cb['callback_selection']['callback_type']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	316 if cb_type not in ALLOWED_CALLBACKS:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	317 raise ValueError(
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	318 "Prohibited callback type: %s!" % cb_type)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	319
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	320 estimator.set_params(**new_params)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	321
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	322 # handle scorer, convert to scorer dict
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	323 scoring = params['experiment_schemes']['metrics']['scoring']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	324 scorer = get_scoring(scoring)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	325 scorer, _ = _check_multimetric_scoring(estimator, scoring=scorer)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	326
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	327 # handle test (first) split
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	328 test_split_options = (params['experiment_schemes']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	329 ['test_split']['split_algos'])
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	330
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	331 if test_split_options['shuffle'] == 'group':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	332 test_split_options['labels'] = groups
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	333 if test_split_options['shuffle'] == 'stratified':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	334 if y is not None:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	335 test_split_options['labels'] = y
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	336 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	337 raise ValueError("Stratified shuffle split is not "
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	338 "applicable on empty target values!")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	339
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	340 X_train, X_test, y_train, y_test, groups_train, groups_test = \
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	341 train_test_split_none(X, y, groups, **test_split_options)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	342
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	343 exp_scheme = params['experiment_schemes']['selected_exp_scheme']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	344
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	345 # handle validation (second) split
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	346 if exp_scheme == 'train_val_test':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	347 val_split_options = (params['experiment_schemes']
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	348 ['val_split']['split_algos'])
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	349
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	350 if val_split_options['shuffle'] == 'group':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	351 val_split_options['labels'] = groups_train
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	352 if val_split_options['shuffle'] == 'stratified':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	353 if y_train is not None:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	354 val_split_options['labels'] = y_train
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	355 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	356 raise ValueError("Stratified shuffle split is not "
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	357 "applicable on empty target values!")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	358
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	359 X_train, X_val, y_train, y_val, groups_train, groups_val = \
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	360 train_test_split_none(X_train, y_train, groups_train,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	361 **val_split_options)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	362
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	363 # train and eval
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	364 if hasattr(estimator, 'validation_data'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	365 if exp_scheme == 'train_val_test':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	366 estimator.fit(X_train, y_train,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	367 validation_data=(X_val, y_val))
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	368 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	369 estimator.fit(X_train, y_train,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	370 validation_data=(X_test, y_test))
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	371 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	372 estimator.fit(X_train, y_train)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	373
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	374 if hasattr(estimator, 'evaluate'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	375 scores = estimator.evaluate(X_test, y_test=y_test,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	376 scorer=scorer,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	377 is_multimetric=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	378 else:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	379 scores = _score(estimator, X_test, y_test, scorer,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	380 is_multimetric=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	381 # handle output
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	382 for name, score in scores.items():
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	383 scores[name] = [score]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	384 df = pd.DataFrame(scores)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	385 df = df[sorted(df.columns)]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	386 df.to_csv(path_or_buf=outfile_result, sep='\t',
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	387 header=True, index=False)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	388
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	389 memory.clear(warn=False)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	390
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	391 if outfile_object:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	392 main_est = estimator
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	393 if isinstance(estimator, pipeline.Pipeline):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	394 main_est = estimator.steps[-1][-1]
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	395
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	396 if hasattr(main_est, 'model_') \
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	397 and hasattr(main_est, 'save_weights'):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	398 if outfile_weights:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	399 main_est.save_weights(outfile_weights)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	400 del main_est.model_
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	401 del main_est.fit_params
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	402 del main_est.model_class_
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	403 del main_est.validation_data
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	404 if getattr(main_est, 'data_generator_', None):
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	405 del main_est.data_generator_
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	406
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	407 with open(outfile_object, 'wb') as output_handler:
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	408 pickle.dump(estimator, output_handler,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	409 pickle.HIGHEST_PROTOCOL)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	410
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	411
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	412 if __name__ == '__main__':
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	413 aparser = argparse.ArgumentParser()
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	414 aparser.add_argument("-i", "--inputs", dest="inputs", required=True)
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	415 aparser.add_argument("-e", "--estimator", dest="infile_estimator")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	416 aparser.add_argument("-X", "--infile1", dest="infile1")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	417 aparser.add_argument("-y", "--infile2", dest="infile2")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	418 aparser.add_argument("-O", "--outfile_result", dest="outfile_result")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	419 aparser.add_argument("-o", "--outfile_object", dest="outfile_object")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	420 aparser.add_argument("-w", "--outfile_weights", dest="outfile_weights")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	421 aparser.add_argument("-g", "--groups", dest="groups")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	422 aparser.add_argument("-r", "--ref_seq", dest="ref_seq")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	423 aparser.add_argument("-b", "--intervals", dest="intervals")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	424 aparser.add_argument("-t", "--targets", dest="targets")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	425 aparser.add_argument("-f", "--fasta_path", dest="fasta_path")
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	426 args = aparser.parse_args()
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	427
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	428 main(args.inputs, args.infile_estimator, args.infile1, args.infile2,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	429 args.outfile_result, outfile_object=args.outfile_object,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	430 outfile_weights=args.outfile_weights, groups=args.groups,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	431 ref_seq=args.ref_seq, intervals=args.intervals,
a36e1455971d planemo upload for repository https://github.com/bgruening/galaxytools/tree/master/tools/sklearn commit 60f0fbc0eafd7c11bc60fb6c77f2937782efd8a9-dirty bgruening parents: diff changeset	432 targets=args.targets, fasta_path=args.fasta_path)

Mercurial > repos > bgruening > sklearn_numeric_clustering

annotate train_test_eval.py @ 36:836ba896e2be draft