Mercurial > repos > peterjc > ncbi_blast_plus
changeset 47:d0de6862cda1 draft
Uploaded v0.1.01, embed citation info, GI and SeqID filters.
line wrap: on
line diff
--- a/tools/ncbi_blast_plus/README.rst Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/README.rst Fri Oct 31 11:43:23 2014 -0400 @@ -1,10 +1,13 @@ Galaxy wrappers for NCBI BLAST+ suite ===================================== -These wrappers are copyright 2010-2013 by Peter Cock (The James Hutton Institute, -UK) and additional contributors. All rights reserved. See the licence text below. +These wrappers are copyright 2010-2014 by Peter Cock (The James Hutton Institute, +UK) and additional contributors including Edward Kirton, John Chilton, +Nicola Soranzo, Jim Johnson, and Bjoern Gruening. -Currently tested with NCBI BLAST 2.2.28+ (i.e. version 2.2.28 of BLAST+), +See the licence text below. + +Currently tested with NCBI BLAST 2.2.29+ (i.e. version 2.2.29 of BLAST+), and does not work with the NCBI 'legacy' BLAST suite (e.g. ``blastall``). Note that these wrappers (and the associated datatypes) were originally @@ -52,8 +55,7 @@ </section> You will also need to install ``blast_datatypes`` from the Tool Shed. This -defines the BLAST XML file format (``blastxml``) and protein and nucleotide -BLAST databases composite file formats (``blastdbp`` and ``blastdbn``): +defines the BLAST XML file format (``blastxml``), BLAST databases, etc: * http://toolshed.g2.bx.psu.edu/view/devteam/blast_datatypes @@ -62,7 +64,7 @@ files. You must install the NCBI BLAST+ standalone tools somewhere on the system -path. Currently the unit tests are written using "BLAST 2.2.29+". +path. Currently the unit tests are written using BLAST 2.2.29+. Run the functional tests (adjusting the section identifier to match your ``tool_conf.xml.sample`` file):: @@ -73,10 +75,10 @@ ============= You must tell Galaxy about any system level BLAST databases using configuration -files blastdb.loc (nucleotide databases like NT) and blastdb_p.loc (protein -databases like NR), and blastdb_d.loc (protein domain databases like CDD or -SMART) which are located in the tool-data/ folder. Sample files are included -which explain the tab-based format to use. +files ``blastdb.loc`` (nucleotide databases like NT) and ``blastdb_p.loc`` +(protein databases like NR), and ``blastdb_d.loc`` (protein domain databases +like CDD or SMART) which are located in the ``tool-data/`` folder. Sample +files are included which explain the tab-based format to use. You can download the NCBI provided databases as tar-balls from here: @@ -130,7 +132,7 @@ get one of the missing columns like query or subject length) v0.0.18 - Defensive quoting of filenames in case of spaces (where possible, BLAST+ handling of some multi-file arguments is problematic). -v0.0.19 - Added wrappers for rpsblast and rpstblastn, and new blastdb_d.loc +v0.0.19 - Added wrappers for rpsblast and rpstblastn, and new ``blastdb_d.loc`` for the domain databases they use (e.g. CDD, PFAM or SMART). - Correct case of exception regular expression (for error handling fall-back in case the return code is not set properly). @@ -177,6 +179,8 @@ (based on contribution from Bjoern Gruening). - The RPS-BLAST and RPS-TBLASTN wrappers support using a protein domain database from the user's history. + - Tool definitions now embed citation information (by John Chilton). + - BLAST tools support GI and SeqID filters (added by Bjoern Gruening). ======= ======================================================================
--- a/tools/ncbi_blast_plus/blastxml_to_tabular.py Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/blastxml_to_tabular.py Fri Oct 31 11:43:23 2014 -0400 @@ -66,7 +66,7 @@ from optparse import OptionParser if "-v" in sys.argv or "--version" in sys.argv: - print "v0.1.00" + print "v0.1.01" sys.exit(0) if sys.version_info[:2] >= ( 2, 5 ): @@ -85,7 +85,20 @@ if len(sys.argv) == 4 and sys.argv[3] in ["std", "x22", "ext"]: #False positive if user really has a BLAST XML file called 'std' or 'ext'... - stop_err("ERROR: The script API has changed, sorry.") + stop_err("""ERROR: The script API has changed, sorry. + +Instead of the old style: + +$ python blastxml_to_tabular.py input.xml output.tabular std + +Please use: + +$ python blastxml_to_tabular.py -o output.tabular -c std input.xml + +For more information, use: + +$ python blastxml_to_tabular.py -h +""") usage = """usage: %prog [options] blastxml[,...]
--- a/tools/ncbi_blast_plus/blastxml_to_tabular.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/blastxml_to_tabular.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="blastxml_to_tabular" name="BLAST XML to tabular" version="0.1.00"> +<tool id="blastxml_to_tabular" name="BLAST XML to tabular" version="0.1.01"> <description>Convert BLAST XML output to tabular</description> <version_command interpreter="python">blastxml_to_tabular.py --version</version_command> <command interpreter="python"> @@ -209,4 +209,7 @@ This wrapper is available to install into other Galaxy Instances via the Galaxy Tool Shed at http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus </help> + <citations> + <citation type="doi">10.7717/peerj.167</citation> + </citations> </tool>
--- a/tools/ncbi_blast_plus/ncbi_blastdbcmd_info.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_blastdbcmd_info.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_blastdbcmd_info" name="NCBI BLAST+ database info" version="0.1.00"> +<tool id="ncbi_blastdbcmd_info" name="NCBI BLAST+ database info" version="0.1.01"> <description>Show BLAST database information from blastdbcmd</description> <macros> <token name="@BINARY@">blastdbcmd</token> @@ -32,4 +32,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_blastdbcmd_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_blastdbcmd_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_blastdbcmd_wrapper" name="NCBI BLAST+ blastdbcmd entry(s)" version="0.1.00"> +<tool id="ncbi_blastdbcmd_wrapper" name="NCBI BLAST+ blastdbcmd entry(s)" version="0.1.01"> <description>Extract sequence(s) from BLAST database</description> <macros> <token name="@BINARY@">blastdbcmd</token> @@ -104,4 +104,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_blastn_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_blastn_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_blastn_wrapper" name="NCBI BLAST+ blastn" version="0.1.00"> +<tool id="ncbi_blastn_wrapper" name="NCBI BLAST+ blastn" version="0.1.01"> <description>Search nucleotide database with nucleotide query sequence(s)</description> <!-- If job splitting is enabled, break up the query file into parts --> <parallelism method="multi" split_inputs="query" split_mode="to_size" split_size="1000" merge_outputs="output1"></parallelism> @@ -24,6 +24,7 @@ -perc_identity $adv_opts.identity_cutoff #end if $adv_opts.ungapped +@ADV_ID_LIST_FILTER@ ## End of advanced options: #end if </command> @@ -62,6 +63,7 @@ </param> <param name="ungapped" type="boolean" label="Perform ungapped alignment only?" truevalue="-ungapped" falsevalue="" checked="false" /> <expand macro="input_parse_deflines" /> + <expand macro="advanced_optional_id_files" /> </expand> </inputs> <outputs> @@ -128,4 +130,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_blastp_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_blastp_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_blastp_wrapper" name="NCBI BLAST+ blastp" version="0.1.00"> +<tool id="ncbi_blastp_wrapper" name="NCBI BLAST+ blastp" version="0.1.01"> <description>Search protein database with protein query sequence(s)</description> <!-- If job splitting is enabled, break up the query file into parts --> <parallelism method="multi" split_inputs="query" split_mode="to_size" split_size="1000" merge_outputs="output1" /> @@ -22,6 +22,7 @@ @ADVANCED_OPTIONS@ ##Ungapped disabled for now - see comments below ##$adv_opts.ungapped +@ADV_ID_LIST_FILTER@ ## End of advanced options: #end if </command> @@ -52,6 +53,7 @@ <param name="ungapped" type="boolean" label="Perform ungapped alignment only?" truevalue="-ungapped -comp_based_stats F" falsevalue="" checked="false" /> --> <expand macro="input_parse_deflines" /> + <expand macro="advanced_optional_id_files" /> </expand> </inputs> <outputs> @@ -144,4 +146,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_blastx_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_blastx_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_blastx_wrapper" name="NCBI BLAST+ blastx" version="0.1.00"> +<tool id="ncbi_blastx_wrapper" name="NCBI BLAST+ blastx" version="0.1.01"> <description>Search protein database with translated nucleotide query sequence(s)</description> <!-- If job splitting is enabled, break up the query file into parts --> <parallelism method="multi" split_inputs="query" split_mode="to_size" split_size="1000" merge_outputs="output1"></parallelism> @@ -22,6 +22,7 @@ -matrix $adv_opts.matrix @ADVANCED_OPTIONS@ $adv_opts.ungapped +@ADV_ID_LIST_FILTER@ ## End of advanced options: #end if </command> @@ -45,6 +46,7 @@ <expand macro="input_word_size" /> <param name="ungapped" type="boolean" label="Perform ungapped alignment only?" truevalue="-ungapped" falsevalue="" checked="false" /> <expand macro="input_parse_deflines" /> + <expand macro="advanced_optional_id_files" /> </expand> </inputs> <outputs> @@ -123,4 +125,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_convert2blastmask_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_convert2blastmask_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_convert2blastmask_wrapper" name="NCBI BLAST+ convert2blastmask" version="0.1.00"> +<tool id="ncbi_convert2blastmask_wrapper" name="NCBI BLAST+ convert2blastmask" version="0.1.01"> <description>Convert masking information in lower-case masked FASTA input to file formats suitable for makeblastdb</description> <macros> <token name="@BINARY@">convert2blastmask</token> @@ -84,4 +84,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_dustmasker_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_dustmasker_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_dustmasker_wrapper" name="NCBI BLAST+ dustmasker" version="0.1.00"> +<tool id="ncbi_dustmasker_wrapper" name="NCBI BLAST+ dustmasker" version="0.1.01"> <!-- dustmasker wrapper from Edward Kirton and Nicola Soranzo --> <description>masks low complexity regions</description> <macros> @@ -96,4 +96,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_macros.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_macros.xml Fri Oct 31 11:43:23 2014 -0400 @@ -113,8 +113,9 @@ <stdio> <!-- Anything other than zero is an error --> <exit_code range="1:" /> + <!-- Might see negative return codes for Unix signals via Python subprocess --> <exit_code range=":-1" /> - <!-- In case the return code has not been set propery check stderr too --> + <!-- In case the return code has not been set properly check stderr too --> <regex match="Error:" /> <regex match="Exception:" /> </stdio> @@ -321,8 +322,42 @@ <when value="advanced"> <yield /> </when> - </conditional> + </conditional> </xml> + <xml name="advanced_optional_id_files"> + <conditional name="adv_optional_id_files_opts"> + <param name="adv_optional_id_files_opts_selector" type="select" + label="Restrict search of database to a given set of ID's" + help="This feature provides a means to exclude ID's from a BLAST database search. The expectation values in the BLAST results are based upon the sequences actually searched, and not on the underlying database. Note this cannot be used when comparing against a FASTA file."> + <option value="none" selected="True">No restriction, search the entire database</option> + <option value="gilist">GI identifers</option> + <option value="negative_gilist">Negative GI identifers</option> + <option value="seqidlist">Sequence identifers (SeqId's)</option> + </param> + <when value="none" /> + <when value="gilist"> + <param name="gilist" type="data" format="txt" label="Restrict search of database to GI's listed in this file" + help="This option is only available for database searches."/> + </when> + <when value="negative_gilist"> + <param name="negative_gilist" type="data" format="txt" label="Restrict search of database to everything except the GI's listed in this file" + help="This option is only available for database searches."/> + </when> + <when value="seqidlist"> + <param name="seqidlist" type="data" format="txt" label=" Restrict search of database to list of SeqId's" + help="This option is only available for database searches."/> + </when> + </conditional> + </xml> + <token name="@ADV_ID_LIST_FILTER@"> +#if $adv_opts.adv_optional_id_files_opts.adv_optional_id_files_opts_selector == 'negative_gilist': + -negative_gilist $adv_opts.adv_optional_id_files_opts.negative_gilist +#elif $adv_opts.adv_optional_id_files_opts.adv_optional_id_files_opts_selector == 'gilist': + -gilist $adv_opts.adv_optional_id_files_opts.gilist +#elif $adv_opts.adv_optional_id_files_opts.adv_optional_id_files_opts_selector == 'seqidlist': + -seqidlist $adv_opts.adv_optional_id_files_opts.seqidlist +#end if + </token> <token name="@THREADS@">-num_threads "\${GALAXY_SLOTS:-8}"</token> <token name="@BLAST_DB_SUBJECT@"> #if $db_opts.db_opts_selector == "db": @@ -379,6 +414,13 @@ This wrapper is available to install into other Galaxy Instances via the Galaxy Tool Shed at http://toolshed.g2.bx.psu.edu/view/devteam/ncbi_blast_plus </token> + <xml name="blast_citations"> + <citations> + <citation type="doi">10.1186/1471-2105-10-421</citation> + <citation type="doi">10.7717/peerj.167</citation> + <!-- TODO: Add BibTeX entry / preprint DOI for Galaxy BLAST+ paper --> + </citations> + </xml> <token name="@OUTPUT_FORMAT@">**Output format** Because Galaxy focuses on processing tabular data, the default output of this
--- a/tools/ncbi_blast_plus/ncbi_makeblastdb.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_makeblastdb.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_makeblastdb" name="NCBI BLAST+ makeblastdb" version="0.1.00"> +<tool id="ncbi_makeblastdb" name="NCBI BLAST+ makeblastdb" version="0.1.01"> <description>Make BLAST database</description> <macros> <token name="@BINARY@">makeblastdb</token> @@ -180,4 +180,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_makeprofiledb.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_makeprofiledb.xml Fri Oct 31 11:43:23 2014 -0400 @@ -124,4 +124,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_rpsblast_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_rpsblast_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -104,4 +104,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_rpstblastn_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_rpstblastn_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -102,4 +102,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_segmasker_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_segmasker_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_segmasker_wrapper" name="NCBI BLAST+ segmasker" version="0.1.00"> +<tool id="ncbi_segmasker_wrapper" name="NCBI BLAST+ segmasker" version="0.1.01"> <description>low-complexity regions in protein sequences</description> <macros> <token name="@BINARY@">segmasker</token> @@ -98,4 +98,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_tblastn_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_tblastn_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_tblastn_wrapper" name="NCBI BLAST+ tblastn" version="0.1.00"> +<tool id="ncbi_tblastn_wrapper" name="NCBI BLAST+ tblastn" version="0.1.01"> <description>Search translated nucleotide database with protein query sequence(s)</description> <!-- If job splitting is enabled, break up the query file into parts --> <parallelism method="multi" split_inputs="query" split_mode="to_size" split_size="1000" merge_outputs="output1"></parallelism> @@ -22,6 +22,7 @@ @ADVANCED_OPTIONS@ ##Ungapped disabled for now - see comments below ##$adv_opts.ungapped +@ADV_ID_LIST_FILTER@ ## End of advanced options: #end if </command> @@ -49,6 +50,7 @@ <param name="ungapped" type="boolean" label="Perform ungapped alignment only?" truevalue="-ungapped -comp_based_stats F" falsevalue="" checked="false" /> --> <expand macro="input_parse_deflines" /> + <expand macro="advanced_optional_id_files" /> </expand> </inputs> <outputs> @@ -158,4 +160,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/ncbi_tblastx_wrapper.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/ncbi_tblastx_wrapper.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ -<tool id="ncbi_tblastx_wrapper" name="NCBI BLAST+ tblastx" version="0.1.00"> +<tool id="ncbi_tblastx_wrapper" name="NCBI BLAST+ tblastx" version="0.1.01"> <description>Search translated nucleotide database with translated nucleotide query sequence(s)</description> <!-- If job splitting is enabled, break up the query file into parts --> <parallelism method="multi" split_inputs="query" split_mode="to_size" split_size="1000" merge_outputs="output1"></parallelism> @@ -24,6 +24,7 @@ ## Need int(str(...)) because $adv_opts.max_hits is an InputValueWrapper object not a string ## Note -max_target_seqs overrides -num_descriptions and -num_alignments @ADVANCED_OPTIONS@ +@ADV_ID_LIST_FILTER@ ## End of advanced options: #end if </command> @@ -49,6 +50,7 @@ <!-- I'd like word_size to be optional, with minimum 2 for tblastx --> <expand macro="input_word_size" /> <expand macro="input_parse_deflines" /> + <expand macro="advanced_optional_id_files" /> </expand> </inputs> <outputs> @@ -74,7 +76,7 @@ **What it does** -Search a *translated nucleotide database* using a *protein query*, +Search a *translated nucleotide database* using a *translated nucleotide query*, using the NCBI BLAST+ tblastx command line tool. @FASTA_WARNING@ @@ -92,4 +94,5 @@ @REFERENCES@ </help> + <expand macro="blast_citations" /> </tool>
--- a/tools/ncbi_blast_plus/repository_dependencies.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/repository_dependencies.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,4 +1,4 @@ <?xml version="1.0"?> <repositories description="This requires the BLAST datatype definitions (e.g. the BLAST XML format)."> - <repository changeset_revision="939a600f45e9" name="blast_datatypes" owner="devteam" toolshed="http://testtoolshed.g2.bx.psu.edu" /> + <repository changeset_revision="939a600f45e9" name="blast_datatypes" owner="devteam" toolshed="https://testtoolshed.g2.bx.psu.edu" /> </repositories>
--- a/tools/ncbi_blast_plus/tool_dependencies.xml Wed Mar 19 10:51:45 2014 -0400 +++ b/tools/ncbi_blast_plus/tool_dependencies.xml Fri Oct 31 11:43:23 2014 -0400 @@ -1,6 +1,6 @@ <?xml version="1.0"?> <tool_dependency> <package name="blast+" version="2.2.29"> - <repository changeset_revision="c021862e9ea8" name="package_blast_plus_2_2_29" owner="iuc" toolshed="http://testtoolshed.g2.bx.psu.edu" /> + <repository changeset_revision="e78bbab7933d" name="package_blast_plus_2_2_29" owner="iuc" toolshed="https://testtoolshed.g2.bx.psu.edu" /> </package> </tool_dependency>