tmhmm_and_signalp: tools/protein_analysis/seq_analysis

annotate tools/protein_analysis/seq_analysis_utils.py @ 10:7c1eda453701 draft

Uploaded revised v0.2.0 with clearer acknowledgement of contributors

author	peterjc
date	Wed, 03 Apr 2013 10:31:27 -0400
parents	5e62aefb2918
children	41a42022f815

rev	line source
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	1 """A few useful functions for working with FASTA files and running jobs.
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	2
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	3 This module was originally written to hold common code used in both the TMHMM
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	4 and SignalP wrappers in Galaxy.
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	5
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	6 Given Galaxy currently supports Python 2.4+ this cannot use the Python module
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	7 multiprocessing so the function run_jobs instead is a simple pool approach
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	8 using just the subprocess library.
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	9 """
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	10 import sys
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	11 import os
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	12 import subprocess
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	13 from time import sleep
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	14
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	15 __version__ = "0.0.1"
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	16
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	17 def stop_err(msg, error_level=1):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	18 """Print error message to stdout and quit with given error level."""
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	19 sys.stderr.write("%s\n" % msg)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	20 sys.exit(error_level)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	21
7 5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	22 try:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	23 from multiprocessing import cpu_count
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	24 except ImportError:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	25 #Must be under Python 2.5, this is copied from multiprocessing:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	26 def cpu_count():
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	27 """Returns the number of CPUs in the system."""
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	28 if sys.platform == 'win32':
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	29 try:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	30 num = int(os.environ['NUMBER_OF_PROCESSORS'])
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	31 except (ValueError, KeyError):
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	32 num = 0
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	33 elif 'bsd' in sys.platform or sys.platform == 'darwin':
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	34 comm = '/sbin/sysctl -n hw.ncpu'
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	35 if sys.platform == 'darwin':
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	36 comm = '/usr' + comm
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	37 try:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	38 with os.popen(comm) as p:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	39 num = int(p.read())
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	40 except ValueError:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	41 num = 0
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	42 else:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	43 try:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	44 num = os.sysconf('SC_NPROCESSORS_ONLN')
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	45 except (ValueError, OSError, AttributeError):
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	46 num = 0
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	47
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	48 if num >= 1:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	49 return num
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	50 else:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	51 raise NotImplementedError('cannot determine number of cpus')
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	52
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	53
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	54 def thread_count(command_line_arg, default=1):
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	55 try:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	56 num = int(command_line_arg)
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	57 except:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	58 num = default
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	59 if num < 1:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	60 stop_err("Threads argument %r is not a positive integer" % command_line_arg)
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	61 #Cap this with the pysical limit of the machine,
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	62 try:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	63 num = min(num, cpu_count())
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	64 except NotImplementedError:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	65 pass
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	66 #For debugging,
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	67 #hostname = os.environ.get("HOSTNAME", "this machine")
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	68 #sys.stderr.write("Using %i cores on %s\n" % (num, hostname))
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	69 return num
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	70
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	71
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	72 def fasta_iterator(filename, max_len=None, truncate=None):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	73 """Simple FASTA parser yielding tuples of (title, sequence) strings."""
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	74 handle = open(filename)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	75 title, seq = "", ""
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	76 for line in handle:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	77 if line.startswith(">"):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	78 if title:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	79 if truncate:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	80 seq = seq[:truncate]
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	81 if max_len and len(seq) > max_len:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	82 raise ValueError("Sequence %s is length %i, max length %i" \
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	83 % (title.split()[0], len(seq), max_len))
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	84 yield title, seq
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	85 title = line[1:].rstrip()
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	86 seq = ""
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	87 elif title:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	88 seq += line.strip()
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	89 elif not line.strip() or line.startswith("#"):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	90 #Ignore blank lines, and any comment lines
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	91 #between records (starting with hash).
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	92 pass
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	93 else:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	94 raise ValueError("Bad FASTA line %r" % line)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	95 handle.close()
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	96 if title:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	97 if truncate:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	98 seq = seq[:truncate]
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	99 if max_len and len(seq) > max_len:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	100 raise ValueError("Sequence %s is length %i, max length %i" \
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	101 % (title.split()[0], len(seq), max_len))
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	102 yield title, seq
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	103 raise StopIteration
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	104
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	105 def split_fasta(input_filename, output_filename_base, n=500, truncate=None, keep_descr=False, max_len=None):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	106 """Split FASTA file into sub-files each of at most n sequences.
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	107
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	108 Returns a list of the filenames used (based on the input filename).
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	109 Each sequence can also be truncated (since we only need the start for
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	110 SignalP), and have its description discarded (since we don't usually
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	111 care about it and some tools don't like very long title lines).
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	112
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	113 If a max_len is given and any sequence exceeds it no temp files are
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	114 created and an exception is raised.
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	115 """
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	116 iterator = fasta_iterator(input_filename, max_len, truncate)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	117 files = []
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	118 try:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	119 while True:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	120 records = []
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	121 for i in range(n):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	122 try:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	123 records.append(iterator.next())
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	124 except StopIteration:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	125 break
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	126 if not records:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	127 break
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	128 new_filename = "%s.%i.tmp" % (output_filename_base, len(files))
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	129 handle = open(new_filename, "w")
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	130 if keep_descr:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	131 for title, seq in records:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	132 handle.write(">%s\n" % title)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	133 for i in range(0, len(seq), 60):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	134 handle.write(seq[i:i+60] + "\n")
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	135 else:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	136 for title, seq in records:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	137 handle.write(">%s\n" % title.split()[0])
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	138 for i in range(0, len(seq), 60):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	139 handle.write(seq[i:i+60] + "\n")
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	140 handle.close()
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	141 files.append(new_filename)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	142 #print "%i records in %s" % (len(records), new_filename)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	143 except ValueError, err:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	144 #Max length failure from parser - clean up
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	145 try:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	146 handle.close()
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	147 except:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	148 pass
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	149 for f in files:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	150 if os.path.isfile(f):
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	151 os.remove(f)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	152 raise err
7 5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	153 for f in files:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	154 assert os.path.isfile(f), "Missing split file %r (!??)" % f
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	155 return files
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	156
6 39a6e46cdda3 Migrated tool version 0.0.9 from old tool shed archive to new tool shed repository peterjc parents: 3 diff changeset	157 def run_jobs(jobs, threads, pause=10, verbose=False):
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	158 """Takes list of cmd strings, returns dict with error levels."""
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	159 pending = jobs[:]
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	160 running = []
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	161 results = {}
7 5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	162 if threads == 1:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	163 #Special case this for speed, don't need the waits
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	164 for cmd in jobs:
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	165 results[cmd] = subprocess.call(cmd, shell=True)
5e62aefb2918 Uploaded v0.1.2 to Test Tool Shed peterjc parents: 6 diff changeset	166 return results
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	167 while pending or running:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	168 #See if any have finished
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	169 for (cmd, process) in running:
3 fe10f448d641 Migrated tool version 0.0.6 from old tool shed archive to new tool shed repository peterjc parents: 0 diff changeset	170 return_code = process.poll() #non-blocking
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	171 if return_code is not None:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	172 results[cmd] = return_code
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	173 running = [(cmd, process) for (cmd, process) in running \
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	174 if cmd not in results]
3 fe10f448d641 Migrated tool version 0.0.6 from old tool shed archive to new tool shed repository peterjc parents: 0 diff changeset	175 if verbose:
fe10f448d641 Migrated tool version 0.0.6 from old tool shed archive to new tool shed repository peterjc parents: 0 diff changeset	176 print "%i jobs pending, %i running, %i completed" \
fe10f448d641 Migrated tool version 0.0.6 from old tool shed archive to new tool shed repository peterjc parents: 0 diff changeset	177 % (len(pending), len(running), len(results))
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	178 #See if we can start any new threads
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	179 while pending and len(running) < threads:
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	180 cmd = pending.pop(0)
3 fe10f448d641 Migrated tool version 0.0.6 from old tool shed archive to new tool shed repository peterjc parents: 0 diff changeset	181 if verbose:
fe10f448d641 Migrated tool version 0.0.6 from old tool shed archive to new tool shed repository peterjc parents: 0 diff changeset	182 print cmd
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	183 process = subprocess.Popen(cmd, shell=True)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	184 running.append((cmd, process))
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	185 #Loop...
6 39a6e46cdda3 Migrated tool version 0.0.9 from old tool shed archive to new tool shed repository peterjc parents: 3 diff changeset	186 sleep(pause)
3 fe10f448d641 Migrated tool version 0.0.6 from old tool shed archive to new tool shed repository peterjc parents: 0 diff changeset	187 if verbose:
fe10f448d641 Migrated tool version 0.0.6 from old tool shed archive to new tool shed repository peterjc parents: 0 diff changeset	188 print "%i jobs completed" % len(results)
0 a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	189 assert set(jobs) == set(results)
a2eeeaa6f75e Migrated tool version 0.0.1 from old tool shed archive to new tool shed repository peterjc parents: diff changeset	190 return results

Mercurial > repos > peterjc > tmhmm_and_signalp

annotate tools/protein_analysis/seq_analysis_utils.py @ 10:7c1eda453701 draft