# HG changeset patch
# User pjbriggs
# Date 1539778467 14400
# Node ID d19fca1c009cc2a547794c3475d19d53af98c11a
# Parent 7b2c40450792b7ec8f1f12666d57137afb72fbcd
planemo upload for repository https://github.com/pjbriggs/Amplicon_analysis-galaxy commit c3970ae16b02774148dbf950880f07255734a9a8-dirty
diff -r 7b2c40450792 -r d19fca1c009c README.rst
--- a/README.rst Wed Oct 17 05:01:57 2018 -0400
+++ b/README.rst Wed Oct 17 08:14:27 2018 -0400
@@ -26,26 +26,8 @@
instance to detect the dependencies and reference data correctly
at run time.
-1. Install the dependencies
----------------------------
-
-If the tool is installed from the Galaxy toolshed (recommended) then
-the dependencies should be installed automatically and this step can
-be skipped.
-
-Otherwise the ``install_tool_deps.sh`` script can be used to fetch and
-install the dependencies locally, for example::
-
- install_tool_deps.sh /path/to/local_tool_dependencies
-
-This can take some time to complete. When finished it should have
-created a set of directories containing the dependencies under the
-specified top level directory.
-
-*NB ``install_tool_deps.sh`` is no longer being maintained*
-
-2. Install the tool files
--------------------------
+1. Install the tool from the toolshed
+-------------------------------------
The core tool is hosted on the Galaxy toolshed, so it can be installed
directly from there (this is the recommended route):
@@ -64,7 +46,7 @@
-3. Install the reference data
+2. Install the reference data
-----------------------------
The script ``References.sh`` from the pipeline package at
@@ -78,33 +60,14 @@
will install the data in ``/path/to/pipeline/data``.
**NB** The final amount of data downloaded and uncompressed will be
-around 6GB.
-
-4. Configure dependencies and reference data in Galaxy
-------------------------------------------------------
-
-The final steps are to make your Galaxy installation aware of the
-tool dependencies and reference data, so it can locate them both when
-the tool is run.
-
-To target the tool dependencies installed previously, add the
-following lines to the ``dependency_resolvers_conf.xml`` file in the
-Galaxy ``config`` directory::
+around 9GB.
-
- ...
-
-
- ...
-
+3. Configure reference data location in Galaxy
+----------------------------------------------
-(NB it is recommended to place these *before* the ````
-resolvers)
-
-(If you're not familiar with dependency resolvers in Galaxy then
-see the documentation at
-https://docs.galaxyproject.org/en/master/admin/dependency_resolvers.html
-for more details.)
+The final step is to make your Galaxy installation aware of the
+location of the reference data, so it can locate them both when the
+tool is run.
The tool locates the reference data via an environment variable called
``AMPLICON_ANALYSIS_REF_DATA_PATH``, which needs to set to the parent
@@ -114,7 +77,8 @@
installation is configured:
* **For local instances:** add a line to set it in the
- ``config/local_env.sh`` file of your Galaxy installation, e.g.::
+ ``config/local_env.sh`` file of your Galaxy installation (you
+ may need to create a new empty file first), e.g.::
export AMPLICON_ANALYSIS_REF_DATA_PATH=/path/to/pipeline/data
@@ -130,9 +94,9 @@
(For more about job destinations see the Galaxy documentation at
- https://galaxyproject.org/admin/config/jobs/#job-destinations)
+ https://docs.galaxyproject.org/en/master/admin/jobs.html#job-destinations)
-5. Enable rendering of HTML outputs from pipeline
+4. Enable rendering of HTML outputs from pipeline
-------------------------------------------------
To ensure that HTML outputs are displayed correctly in Galaxy
@@ -177,6 +141,33 @@
https://github.com/galaxyproject/galaxy/issues/4490 and
https://github.com/galaxyproject/galaxy/issues/1676
+Appendix: installing the dependencies manually
+==============================================
+
+If the tool is installed from the Galaxy toolshed (recommended) then
+the dependencies should be installed automatically and this step can
+be skipped.
+
+Otherwise the ``install_amplicon_analysis_deps.sh`` script can be used
+to fetch and install the dependencies locally, for example::
+
+ install_amplicon_analysis.sh /path/to/local_tool_dependencies
+
+(This is the same script as is used to install dependencies from the
+toolshed.) This can take some time to complete, and when completed will
+have created a directory called ``Amplicon_analysis-1.2.3`` containing
+the dependencies under the specified top level directory.
+
+**NB** The installed dependencies will occupy around 2.6G of disk
+space.
+
+You will need to make sure that the ``bin`` subdirectory of this
+directory is on Galaxy's ``PATH`` at runtime, for the tool to be able
+to access the dependencies - for example by adding a line to the
+``local_env.sh`` file like::
+
+ export PATH=/path/to/local_tool_dependencies/Amplicon_analysis-1.2.3/bin:$PATH
+
History
=======
@@ -184,7 +175,7 @@
Version Changes
---------- ----------------------------------------------------------------------
1.2.3.0 Updated to Amplicon_Analysis_Pipeline version 1.2.3; install
- dependencies from bioconda and via tool_dependencies.xml.
+ dependencies via tool_dependencies.xml.
1.2.2.0 Updated to Amplicon_Analysis_Pipeline version 1.2.2 (removes
jackknifed analysis which is not captured by Galaxy tool)
1.2.1.0 Updated to Amplicon_Analysis_Pipeline version 1.2.1 (adds
diff -r 7b2c40450792 -r d19fca1c009c amplicon_analysis_pipeline.py
--- a/amplicon_analysis_pipeline.py Wed Oct 17 05:01:57 2018 -0400
+++ b/amplicon_analysis_pipeline.py Wed Oct 17 08:14:27 2018 -0400
@@ -97,15 +97,6 @@
if filen is not None:
fp.close()
-def find_executable(name):
- # Locate executable on PATH
- for p in os.environ['PATH'].split(os.pathsep):
- exe = os.path.normpath(
- os.path.abspath(os.path.join(p,name)))
- if os.path.isfile(exe) and os.access(exe,os.X_OK):
- return exe
- return None
-
if __name__ == "__main__":
# Command line
print "Amplicon analysis: starting"
@@ -149,11 +140,12 @@
# Link to FASTQs and construct Final_name.txt file
sample_names = []
+ print "-- making Final_name.txt"
with open("Final_name.txt",'w') as final_name:
fastqs = iter(args.fastq_pairs)
for sample_name,fqr1,fqr2 in zip(fastqs,fastqs,fastqs):
sample_name = clean_up_name(sample_name)
- print "Cleaned up sample name: %s" % sample_name
+ print " %s" % sample_name
r1 = "%s_R1_.fastq" % sample_name
r2 = "%s_R2_.fastq" % sample_name
os.symlink(fqr1,r1)
diff -r 7b2c40450792 -r d19fca1c009c amplicon_analysis_pipeline.xml
--- a/amplicon_analysis_pipeline.xml Wed Oct 17 05:01:57 2018 -0400
+++ b/amplicon_analysis_pipeline.xml Wed Oct 17 08:14:27 2018 -0400
@@ -1,7 +1,7 @@
analyse 16S rRNA data from Illumina Miseq paired-end reads
- amplicon_analysis_pipeline-all
+ amplicon_analysis_pipeline
diff -r 7b2c40450792 -r d19fca1c009c install_amplicon_analysis.sh
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/install_amplicon_analysis.sh Wed Oct 17 08:14:27 2018 -0400
@@ -0,0 +1,425 @@
+#!/bin/sh -e
+#
+# Prototype script to setup a conda environment with the
+# dependencies needed for the Amplicon_analysis_pipeline
+# script
+#
+# Handle command line
+usage()
+{
+ echo "Usage: $(basename $0) [DIR]"
+ echo ""
+ echo "Installs the Amplicon_analysis_pipeline package plus"
+ echo "dependencies in directory DIR (or current directory "
+ echo "if DIR not supplied)"
+}
+if [ ! -z "$1" ] ; then
+ # Check if help was requested
+ case "$1" in
+ --help|-h)
+ usage
+ exit 0
+ ;;
+ esac
+ # Assume it's the installation directory
+ cd $1
+fi
+# Versions
+PIPELINE_VERSION=1.2.3
+RDP_CLASSIFIER_VERSION=2.2
+# Directories
+TOP_DIR=$(pwd)/Amplicon_analysis-${PIPELINE_VERSION}
+BIN_DIR=${TOP_DIR}/bin
+CONDA_DIR=${TOP_DIR}/conda
+CONDA_BIN=${CONDA_DIR}/bin
+CONDA_LIB=${CONDA_DIR}/lib
+CONDA=${CONDA_BIN}/conda
+ENV_NAME="amplicon_analysis_pipeline@${PIPELINE_VERSION}"
+ENV_DIR=${CONDA_DIR}/envs/$ENV_NAME
+#
+# Functions
+#
+# Report failure and terminate script
+fail()
+{
+ echo ""
+ echo ERROR $@ >&2
+ echo ""
+ echo "$(basename $0): installation failed"
+ exit 1
+}
+#
+# Rewrite the shebangs in the installed conda scripts
+# to remove the full path to conda 'bin' directory
+rewrite_conda_shebangs()
+{
+ pattern="s,^#!${CONDA_BIN}/,#!/usr/bin/env ,g"
+ find ${CONDA_BIN} -type f -exec sed -i "$pattern" {} \;
+}
+#
+# Install conda
+install_conda()
+{
+ echo "++++++++++++++++"
+ echo "Installing conda"
+ echo "++++++++++++++++"
+ if [ -e ${CONDA_DIR} ] ; then
+ echo "*** $CONDA_DIR already exists ***" >&2
+ return
+ fi
+ local cwd=$(pwd)
+ local wd=$(mktemp -d)
+ cd $wd
+ wget -q https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh
+ bash ./Miniconda2-latest-Linux-x86_64.sh -b -p ${CONDA_DIR}
+ echo Installed conda in ${CONDA_DIR}
+ # Update the installation files
+ # This is to avoid problems when the length the installation
+ # directory path exceeds the limit for the shebang statement
+ # in the conda files
+ echo ""
+ echo -n "Rewriting conda shebangs..."
+ rewrite_conda_shebangs
+ echo "ok"
+ echo -n "Adding conda bin to PATH..."
+ PATH=${CONDA_BIN}:$PATH
+ echo "ok"
+ cd $cwd
+ rm -rf $wd/*
+ rmdir $wd
+}
+#
+# Create conda environment
+install_conda_packages()
+{
+ echo "+++++++++++++++++++++++++"
+ echo "Installing conda packages"
+ echo "+++++++++++++++++++++++++"
+ local cwd=$(pwd)
+ local wd=$(mktemp -d)
+ cd $wd
+ cat >environment.yml <${BIN_DIR}/Amplicon_analysis_pipeline.sh <${BIN_DIR}/install_reference_data.sh <${BIN_DIR}/ChimeraSlayer.pl <INSTALL.log 2>&1
+ echo "ok"
+ cd R-3.2.1
+ echo -n "Running configure..."
+ ./configure --prefix=$INSTALL_DIR --with-x=no --with-readline=no >>INSTALL.log 2>&1
+ echo "ok"
+ echo -n "Running make..."
+ make >>INSTALL.log 2>&1
+ echo "ok"
+ echo -n "Running make install..."
+ make install >>INSTALL.log 2>&1
+ echo "ok"
+ cd $cwd
+ rm -rf $wd/*
+ rmdir $wd
+ . ${CONDA_BIN}/deactivate
+}
+setup_pipeline_environment()
+{
+ echo "+++++++++++++++++++++++++++++++"
+ echo "Setting up pipeline environment"
+ echo "+++++++++++++++++++++++++++++++"
+ # vsearch113
+ echo -n "Setting up vsearch113..."
+ if [ -e ${BIN_DIR}/vsearch113 ] ; then
+ echo "already exists"
+ elif [ ! -e ${ENV_DIR}/bin/vsearch ] ; then
+ echo "failed"
+ fail "vsearch not found"
+ else
+ ln -s ${ENV_DIR}/bin/vsearch ${BIN_DIR}/vsearch113
+ echo "ok"
+ fi
+ # fasta_splitter.pl
+ echo -n "Setting up fasta_splitter.pl..."
+ if [ -e ${BIN_DIR}/fasta-splitter.pl ] ; then
+ echo "already exists"
+ elif [ ! -e ${ENV_DIR}/share/fasta-splitter/fasta-splitter.pl ] ; then
+ echo "failed"
+ fail "fasta-splitter.pl not found"
+ else
+ ln -s ${ENV_DIR}/share/fasta-splitter/fasta-splitter.pl ${BIN_DIR}/fasta-splitter.pl
+ echo "ok"
+ fi
+ # rdp_classifier.jar
+ local rdp_classifier_jar=rdp_classifier-${RDP_CLASSIFIER_VERSION}.jar
+ echo -n "Setting up rdp_classifier.jar..."
+ if [ -e ${TOP_DIR}/share/rdp_classifier/${rdp_classifier_jar} ] ; then
+ echo "already exists"
+ elif [ ! -e ${ENV_DIR}/share/rdp_classifier/rdp_classifier.jar ] ; then
+ echo "failed"
+ fail "rdp_classifier.jar not found"
+ else
+ mkdir -p ${TOP_DIR}/share/rdp_classifier
+ ln -s ${ENV_DIR}/share/rdp_classifier/rdp_classifier.jar ${TOP_DIR}/share/rdp_classifier/${rdp_classifier_jar}
+ echo "ok"
+ fi
+ # qiime_config
+ echo -n "Setting up qiime_config..."
+ if [ -e ${TOP_DIR}/qiime/qiime_config ] ; then
+ echo "already exists"
+ else
+ mkdir -p ${TOP_DIR}/qiime
+ cat >${TOP_DIR}/qiime/qiime_config <>$INSTALL_DIR/INSTALLATION.log 2>&1
-EOF
- popd
- rm -rf $wd/*
- rmdir $wd
-}
-function install_amplicon_analysis_pipeline_1_2_2() {
- install_amplicon_analysis_pipeline $1 1.2.2
-}
-function install_amplicon_analysis_pipeline_1_2_1() {
- install_amplicon_analysis_pipeline $1 1.2.1
-}
-function install_amplicon_analysis_pipeline_1_1() {
- install_amplicon_analysis_pipeline $1 1.1
-}
-function install_amplicon_analysis_pipeline_1_0() {
- install_amplicon_analysis_pipeline $1 1.0
-}
-function install_amplicon_analysis_pipeline() {
- version=$2
- echo Installing Amplicon_analysis $version
- install_dir=$1/amplicon_analysis_pipeline/$version
- if [ -f $install_dir/env.sh ] ; then
- return
- fi
- mkdir -p $install_dir
- echo Moving to $install_dir
- pushd $install_dir
- wget -q https://github.com/MTutino/Amplicon_analysis/archive/v${version}.tar.gz
- tar zxf v${version}.tar.gz
- mv Amplicon_analysis-${version} Amplicon_analysis
- rm -rf v${version}.tar.gz
- popd
- # Make setup file
- cat > $install_dir/env.sh < $install_dir/env.sh < $INSTALL_DIR/env.sh <$INSTALL_DIR/INSTALLATION.log 2>&1
- mv sickle $INSTALL_DIR/bin
- popd
- rm -rf $wd/*
- rmdir $wd
- # Make setup file
- cat > $INSTALL_DIR/env.sh <$INSTALL_DIR/INSTALLATION.log 2>&1
- mv bioawk $INSTALL_DIR/bin
- mv maketab $INSTALL_DIR/bin
- popd
- rm -rf $wd/*
- rmdir $wd
- # Make setup file
- cat > $INSTALL_DIR/env.sh <$install_dir/INSTALLATION.log 2>&1
- ./configure --prefix=$install_dir >>$install_dir/INSTALLATION.log 2>&1
- make; make install >>$install_dir/INSTALLATION.log 2>&1
- popd
- rm -rf $wd/*
- rmdir $wd
- # Make setup file
- cat > $1/pandaseq/2.8.1/env.sh < $1/spades/3.5.0/env.sh < $1/fastqc/0.11.3/env.sh < test.f90
- gfortran -o test test.f90
- LGF=`ldd test | grep libgfortran | awk '{print $3}'`
- LGF_CANON=`readlink -f $LGF`
- LGF_VERS=`objdump -p $LGF_CANON | grep GFORTRAN_1 | sed -r 's/.*GFORTRAN_1\.([0-9])+/\1/' | sort -n | tail -1`
- if [ $LGF_VERS -gt $BUNDLED_LGF_VERS ]; then
- cp -p $BUNDLED_LGF_CANON ${BUNDLED_LGF_CANON}.bundled
- cp -p $LGF_CANON $BUNDLED_LGF_CANON
- fi
- popd
- rm -rf $wd/*
- rmdir $wd
- # Atlas 3.10 (build from source)
- # NB this stolen from galaxyproject/iuc-tools
- ##local wd=$(mktemp -d)
- ##echo Moving to $wd
- ##pushd $wd
- ##wget -q https://depot.galaxyproject.org/software/atlas/atlas_3.10.2+gx0_src_all.tar.bz2
- ##wget -q https://depot.galaxyproject.org/software/lapack/lapack_3.5.0_src_all.tar.gz
- ##wget -q https://depot.galaxyproject.org/software/atlas/atlas_patch-blas-lapack-1.0_src_all.diff
- ##wget -q https://depot.galaxyproject.org/software/atlas/atlas_patch-shared-lib-1.0_src_all.diff
- ##wget -q https://depot.galaxyproject.org/software/atlas/atlas_patch-cpu-throttle-1.0_src_all.diff
- ##tar -jxvf atlas_3.10.2+gx0_src_all.tar.bz2
- ##cd ATLAS
- ##mkdir build
- ##patch -p1 < ../atlas_patch-blas-lapack-1.0_src_all.diff
- ##patch -p1 < ../atlas_patch-shared-lib-1.0_src_all.diff
- ##patch -p1 < ../atlas_patch-cpu-throttle-1.0_src_all.diff
- ##cd build
- ##../configure --prefix="$INSTALL_DIR" -D c -DWALL -b 64 -Fa alg '-fPIC' --with-netlib-lapack-tarfile=../../lapack_3.5.0_src_all.tar.gz -v 2 -t 0 -Si cputhrchk 0
- ##make
- ##make install
- ##popd
- ##rm -rf $wd/*
- ##rmdir $wd
- export ATLAS_LIB_DIR=$INSTALL_DIR/lib
- export ATLAS_INCLUDE_DIR=$INSTALL_DIR/include
- export ATLAS_BLAS_LIB_DIR=$INSTALL_DIR/lib/atlas
- export ATLAS_LAPACK_LIB_DIR=$INSTALL_DIR/lib/atlas
- export ATLAS_ROOT_PATH=$INSTALL_DIR
- export LD_LIBRARY_PATH=$INSTALL_DIR/lib:$LD_LIBRARY_PATH
- export LD_LIBRARY_PATH=$INSTALL_DIR/lib/atlas:$LD_LIBRARY_PATH
- # Numpy 1.7.1
- local wd=$(mktemp -d)
- echo Moving to $wd
- pushd $wd
- wget -q https://depot.galaxyproject.org/software/numpy/numpy_1.7_src_all.tar.gz
- tar -zxvf numpy_1.7_src_all.tar.gz
- cd numpy-1.7.1
- cat > site.cfg < $INSTALL_DIR/env.sh < $install_dir/env.sh <$install_dir/INSTALLATION.log 2>&1
- mv * $install_dir
- popd
- # Clean up
- rm -rf $wd/*
- rmdir $wd
- # Make setup file
-cat > $install_dir/env.sh < $install_dir/env.sh < $install_dir/env.sh <>$install_dir/INSTALLATION.log
-EOF
- done
- # Install fasta-splitter
- wget -q http://kirill-kryukov.com/study/tools/fasta-splitter/files/fasta-splitter-0.2.4.zip
- unzip -qq fasta-splitter-0.2.4.zip
- chmod 0755 fasta-splitter.pl
- mv fasta-splitter.pl $install_dir/bin
- popd
- # Clean up
- rm -rf $wd/*
- rmdir $wd
- # Make setup file
-cat > $install_dir/env.sh < $install_dir/env.sh < $install_dir/env.sh <$install_dir/bin/uc2otutab.py
- cat uc2otutab.py >>$install_dir/bin/uc2otutab.py
- chmod +x $install_dir/bin/uc2otutab.py
- popd
- # Clean up
- rm -rf $wd/*
- rmdir $wd
- # Make setup file
-cat > $install_dir/env.sh <
-
+
- https://raw.githubusercontent.com/pjbriggs/Amplicon_analysis-galaxy/5fe08ecabf682265778d6307fa63a44c768d251c/install_amplicon_analysis.sh
+ https://raw.githubusercontent.com/pjbriggs/Amplicon_analysis-galaxy/master/amplicon_analysis_pipeline.py
sh ./install_amplicon_analysis.sh $INSTALL_DIR