Mercurial > repos > devteam > column_maker

annotate column_maker.py @ 8:227e82286a0e draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
"planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
author devteam
date Wed, 24 Feb 2021 05:19:12 +0000
parents e7c273e8d4d6
children 33b81f9ea109
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
1 #!/usr/bin/env python
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
2 """
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
3 This tool takes a tab-delimited textfile as input and creates another column in
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
4 the file which is the result of a computation performed on every row in the
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
5 original file. The tool will skip over invalid lines within the file,
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
6 informing the user about the number of lines skipped.
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
7 """
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
8
6
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
9 import argparse
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
10 import json
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
11 import re
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
12 import sys
8
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
13 # functions that may be used in the compute expression
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
14 from math import ( # noqa: F401
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
15 ceil,
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
16 exp,
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
17 floor,
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
18 log,
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
19 log10,
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
20 sqrt
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
21 )
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
22
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
23 from numpy import format_float_positional # noqa: F401
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
24
6
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
25 parser = argparse.ArgumentParser()
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
26 parser.add_argument('input', type=argparse.FileType('r'), help="input file")
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
27 parser.add_argument('output', type=argparse.FileType('wt'), help="output file")
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
28 parser.add_argument('cond', nargs='?', type=str, help="expression")
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
29 parser.add_argument('columns', nargs='?', type=int, help="number of columns")
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
30 parser.add_argument('column_types', nargs='?', type=str, help="comma separated list of column types")
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
31 parser.add_argument('--round', action="store_true",
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
32 help="round result")
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
33 parser.add_argument('--avoid_scientific_notation', action="store_true",
6
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
34 help="avoid scientific notation")
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
35 parser.add_argument('--header_new_column_name', default=None, type=str,
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
36 help="First line of input is a header line with column "
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
37 "names and this should become the name of the new "
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
38 "column")
6
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
39 parser.add_argument('--load_json', default=None, type=argparse.FileType('r'),
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
40 help="overwrite parsed arguments from json file")
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
41 args = parser.parse_args()
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
42
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
43 argparse_dict = vars(args)
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
44 if args.load_json:
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
45 json_dict = json.load(args.load_json)
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
46 argparse_dict.update(json_dict)
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
47
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
48 fh = argparse_dict['input']
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
49 out = argparse_dict['output']
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
50 expr = argparse_dict['cond']
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
51 round_result = argparse_dict['round']
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
52 avoid_scientific_notation = argparse_dict['avoid_scientific_notation']
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
53
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
54 if argparse_dict['header_new_column_name'] is not None:
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
55 header_line = fh.readline().strip('\n')
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
56 out.write(
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
57 '{0}\t{1}\n'.format(
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
58 header_line, argparse_dict['header_new_column_name']
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
59 )
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
60 )
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
61 try:
6
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
62 in_columns = int(argparse_dict['columns'])
8
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
63 if in_columns < 1:
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
64 # To be considered tabular, data must have at least one column.
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
65 raise ValueError
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
66 except Exception:
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
67 if not fh.readline():
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
68 # empty file content is ok and should produce empty output
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
69 out.close()
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
70 sys.exit()
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
71 sys.exit("Missing or invalid 'columns' metadata value, click the pencil icon in the history item and select the Auto-detect option to correct it. This tool can only be used with tab-delimited data.")
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
72 try:
6
0aeda7a81b46 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit a993d43d9d1702a6cf584683cf72527a3f999236"
devteam
parents: 5
diff changeset
73 in_column_types = argparse_dict['column_types'].split(',')
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
74 except Exception:
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
75 sys.exit("Missing or invalid 'column_types' metadata value, click the pencil icon in the history item and select the Auto-detect option to correct it. This tool can only be used with tab-delimited data.")
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
76 if len(in_column_types) != in_columns:
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
77 sys.exit("The 'columns' metadata setting does not conform to the 'column_types' metadata setting, click the pencil icon in the history item and select the Auto-detect option to correct it. This tool can only be used with tab-delimited data.")
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
78
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
79 operators = 'is|not|or|and'
5
c6fdc1118036 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 0cb87d8f454f205af021d653d7f7d5a7c14c7718"
devteam
parents: 4
diff changeset
80 builtin_and_math_functions = 'abs|all|any|bin|chr|cmp|complex|divmod|float|bool|hex|int|len|long|max|min|oct|ord|pow|range|reversed|round|sorted|str|sum|type|unichr|unicode|log|log10|exp|sqrt|ceil|floor'
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
81 string_and_list_methods = [name for name in dir('') + dir([]) if not name.startswith('_')]
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
82 whitelist = r"^([c0-9\+\-\*\/\(\)\.\'\"><=,:! ]|%s|%s|%s)*$" % (operators, builtin_and_math_functions, '|'.join(string_and_list_methods))
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
83 if not re.compile(whitelist).match(expr):
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
84 sys.exit("Invalid expression")
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
85 if avoid_scientific_notation:
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
86 expr = "format_float_positional(%s)" % expr
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
87
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
88 # Prepare the column variable names and wrappers for column data types
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
89 cols, type_casts = [], []
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
90 for col in range(1, in_columns + 1):
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
91 col_name = "c%d" % col
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
92 cols.append(col_name)
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
93 col_type = in_column_types[col - 1].strip()
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
94 if not round_result and col_type == 'int':
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
95 col_type = 'float'
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
96 type_cast = "%s(%s)" % (col_type, col_name)
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
97 type_casts.append(type_cast)
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
98
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
99 col_str = ', '.join(cols) # 'c1, c2, c3, c4'
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
100 type_cast_str = ', '.join(type_casts) # 'str(c1), int(c2), int(c3), str(c4)'
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
101 assign = "%s = line.split('\\t')" % col_str
8
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
102 if len(cols) == 1:
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
103 # Single column, unpacking by assignment won't work
227e82286a0e "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit c623e4b9f6dd6ff4b88d9386f3b0a0cb497e0a0e"
devteam
parents: 7
diff changeset
104 assign += '[0]'
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
105 wrap = "%s = %s" % (col_str, type_cast_str)
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
106 skipped_lines = 0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
107 first_invalid_line = 0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
108 invalid_line = None
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
109 lines_kept = 0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
110 total_lines = 0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
111
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
112 # Read input file, skipping invalid lines, and perform computation that will result in a new column
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
113 code = '''
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
114 for i, line in enumerate(fh):
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
115 total_lines += 1
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
116 line = line.rstrip('\\r\\n')
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
117 if not line or line.startswith('#'):
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
118 skipped_lines += 1
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
119 if not invalid_line:
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
120 first_invalid_line = i + 1
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
121 invalid_line = line
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
122 continue
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
123 try:
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
124 %s
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
125 %s
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
126 new_val = %s
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
127 if round_result:
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
128 new_val = int(round(new_val))
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
129 new_line = line + '\\t' + str(new_val) + "\\n"
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
130 out.write(new_line)
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
131 lines_kept += 1
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
132 except Exception:
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
133 skipped_lines += 1
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
134 if not invalid_line:
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
135 first_invalid_line = i + 1
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
136 invalid_line = line
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
137 fh.close()
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
138 ''' % (assign, wrap, expr)
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
139
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
140 valid_expr = True
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
141 try:
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
142 exec(code)
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
143 except Exception as e:
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
144 out.close()
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
145 if str(e).startswith('invalid syntax'):
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
146 valid_expr = False
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
147 sys.exit('Expression "%s" likely invalid. See tool tips, syntax and examples.' % expr)
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
148 else:
7
e7c273e8d4d6 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 35c79ccee143e8d178ebd24f6317888de3ca0187"
devteam
parents: 6
diff changeset
149 sys.exit(str(e))
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
150
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
151 if valid_expr:
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
152 out.close()
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
153 valid_lines = total_lines - skipped_lines
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
154 print('Creating column %d with expression %s' % (in_columns + 1, expr))
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
155 if valid_lines > 0:
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
156 print('kept %4.2f%% of %d lines.' % (100.0 * lines_kept / valid_lines,
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
157 total_lines))
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
158 else:
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
159 print('Possible invalid expression "%s" or non-existent column referenced. See tool tips, syntax and examples.' % expr)
0
4dadd821722f Imported from capsule None
devteam
parents:
diff changeset
160 if skipped_lines > 0:
4
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
161 print('Skipped %d invalid lines starting at line #%d: "%s"' %
3335c0d583d8 "planemo upload for repository https://github.com/galaxyproject/tools-devteam/tree/master/tools/column_maker commit 2b17bdfc47ca4d7f1a584216c4bd61a7050df7ea"
devteam
parents: 2
diff changeset
162 (skipped_lines, first_invalid_line, invalid_line))