Mercurial > repos > pitagora > unix_tools
changeset 2:8fd5091a4091 default tip
commit
author | pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp> |
---|---|
date | Sun, 19 Oct 2014 17:20:13 +0900 |
parents | 69fbeac761c2 |
children | |
files | awk.xml sed.xml sed_wrapper.sh |
diffstat | 3 files changed, 143 insertions(+), 1 deletions(-) [+] |
line wrap: on
line diff
--- a/awk.xml Sun Oct 19 15:07:18 2014 +0900 +++ b/awk.xml Sun Oct 19 17:20:13 2014 +0900 @@ -1,5 +1,5 @@ <!-- - This tool is based on 'cshl_awk_tool' from Hannon Lab CSHL + This tool is based on 'cshl_awk_tool' from Hannon Lab, CSHL: http://hannonlab.cshl.edu/galaxy_unix_tools/ Thanks. Pitagora -->
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/sed.xml Sun Oct 19 17:20:13 2014 +0900 @@ -0,0 +1,104 @@ +<!-- + This tool is based on 'cshl_sed_tool' from Hannon Lab, CSHL: + http://hannonlab.cshl.edu/galaxy_unix_tools/ + Thanks. Pitagora +--> +<tool id="sed" name="sed"> + <description></description> + <!-- NOTE + 'sandbox' is a patched SED program, + which blocks executing shell commands and file reading/writing. + + Hopefully, it is safe enough to allow users to execute their own SED commands + --> + <command interpreter="sh">sed_wrapper.sh $silent $input $output '$url_paste'</command> + <inputs> + <param format="txt" name="input" type="data" label="File to process" /> + + <!-- Note: the parameter nane MUST BE 'url_paste' - + This is a hack in the galaxy library (see ./lib/galaxy/util/__init__.py line 142) + If the name is 'url_paste' the string won't be sanitized, and all the non-alphanumeric characters + will be passed to the shell script --> + <param name="url_paste" type="text" area="true" size="5x35" label="SED Program" help=""> + <sanitizer> + <valid initial="string.printable"> + <remove value="'"/> + </valid> + <mapping initial="none"> + <add source="'" target="__sq__"/> + </mapping> + </sanitizer> + </param> + + <param name="silent" type="select" label="operation mode" help="(Same as 'sed -n', leave at 'normal' unless you know what you're doing)" > + <option value="">normal</option> + <option value="-n">silent</option> + </param> + + </inputs> + <outputs> + <data format="input" name="output" metadata_source="input" /> + </outputs> +<help> + +**What it does** + +This tool runs the unix **sed** command on the selected data file. + +.. class:: infomark + +**TIP:** This tool uses the **extended regular** expression syntax (same as running 'sed -r'). + + + +**Further reading** + +- Short sed tutorial (http://www.linuxhowtos.org/System/sed_tutorial.htm) +- Long sed tutorial (http://www.grymoire.com/Unix/Sed.html) +- sed faq with good examples (http://sed.sourceforge.net/sedfaq.html) +- sed cheat-sheet (http://www.catonmat.net/download/sed.stream.editor.cheat.sheet.pdf) +- Collection of useful sed one-liners (http://student.northpark.edu/pemente/sed/sed1line.txt) + +----- + +**Sed commands** + +The most useful sed command is **s** (substitute). + +**Examples** + +- **s/hsa//** will remove the first instance of 'hsa' in every line. +- **s/hsa//g** will remove all instances (beacuse of the **g**) of 'hsa' in every line. +- **s/A{4,}/--&--/g** will find sequences of 4 or more consecutive A's, and once found, will surround them with two dashes from each side. The **&** marker is a place holder for 'whatever matched the regular expression'. +- **s/hsa-mir-([^ ]+)/short name: \\1 full name: &/** will find strings such as 'hsa-mir-43a' (the regular expression is 'hsa-mir-' followed by non-space characters) and will replace it will string such as 'short name: 43a full name: hsa-mir-43a'. The **\\1** marker is a place holder for 'whatever matched the first parenthesis' (similar to perl's **$1**) . + + +**sed's Regular Expression Syntax** + +The select tool searches the data for lines containing or not containing a match to the given pattern. A Regular Expression is a pattern descibing a certain amount of text. + +- **( ) { } [ ] . * ? + \ ^ $** are all special characters. **\\** can be used to "escape" a special character, allowing that special character to be searched for. +- **^** matches the beginning of a string(but not an internal line). +- **(** .. **)** groups a particular pattern. +- **{** n or n, or n,m **}** specifies an expected number of repetitions of the preceding pattern. + + - **{n}** The preceding item is matched exactly n times. + - **{n,}** The preceding item ismatched n or more times. + - **{n,m}** The preceding item is matched at least n times but not more than m times. + +- **[** ... **]** creates a character class. Within the brackets, single characters can be placed. A dash (-) may be used to indicate a range such as **a-z**. +- **.** Matches any single character except a newline. +- ***** The preceding item will be matched zero or more times. +- **?** The preceding item is optional and matched at most once. +- **+** The preceding item will be matched one or more times. +- **^** has two meaning: + - matches the beginning of a line or string. + - indicates negation in a character class. For example, [^...] matches every character except the ones inside brackets. +- **$** matches the end of a line or string. +- **\|** Separates alternate possibilities. + + +**Note**: SED uses extended regular expression syntax, not Perl syntax. **\\d**, **\\w**, **\\s** etc. are **not** supported. + +</help> +</tool>
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/sed_wrapper.sh Sun Oct 19 17:20:13 2014 +0900 @@ -0,0 +1,38 @@ +#!/bin/sh + +## +## Galaxy wrapper for SED command +## + +## +## command line arguments: +## input_file +## output_file +## sed-program +## [other parameters passed on to sed] + +INPUT="$1" +OUTPUT="$2" +PROG="$3" + +shift 3 + +if [ -z "$PROG" ]; then + echo usage: $0 INPUTFILE OUTPUTFILE SED-PROGRAM [other sed patameters] >&2 + exit 1 +fi + +if [ ! -r "$INPUT" ]; then + echo "error: input file ($INPUT) not found!" >&2 + exit 1 +fi + +# Messages printed to STDOUT will be displayed in the "INFO" field in the galaxy dataset. +# This way the user can tell what was the command +echo "sed" "$@" "$PROG" + +#$GALAXY_TOOLS/sed/sed/sed -r --sandbox "$@" "$PROG" "$INPUT" > "$OUTPUT" +sed -r "$@" "$PROG" "$INPUT" > "$OUTPUT" +if (( $? )); then exit; fi + +exit 0