Mercurial > repos > pitagora > unix_tools

annotate sed.xml @ 2:8fd5091a4091 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
commit
author pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
date Sun, 19 Oct 2014 17:20:13 +0900
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
1 <!--
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
2 This tool is based on 'cshl_sed_tool' from Hannon Lab, CSHL:
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
3 http://hannonlab.cshl.edu/galaxy_unix_tools/
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
4 Thanks. Pitagora
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
5 -->
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
6 <tool id="sed" name="sed">
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
7 <description></description>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
8 <!-- NOTE
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
9 'sandbox' is a patched SED program,
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
10 which blocks executing shell commands and file reading/writing.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
11
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
12 Hopefully, it is safe enough to allow users to execute their own SED commands
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
13 -->
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
14 <command interpreter="sh">sed_wrapper.sh $silent $input $output '$url_paste'</command>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
15 <inputs>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
16 <param format="txt" name="input" type="data" label="File to process" />
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
17
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
18 <!-- Note: the parameter nane MUST BE 'url_paste' -
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
19 This is a hack in the galaxy library (see ./lib/galaxy/util/__init__.py line 142)
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
20 If the name is 'url_paste' the string won't be sanitized, and all the non-alphanumeric characters
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
21 will be passed to the shell script -->
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
22 <param name="url_paste" type="text" area="true" size="5x35" label="SED Program" help="">
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
23 <sanitizer>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
24 <valid initial="string.printable">
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
25 <remove value="&apos;"/>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
26 </valid>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
27 <mapping initial="none">
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
28 <add source="&apos;" target="__sq__"/>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
29 </mapping>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
30 </sanitizer>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
31 </param>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
32
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
33 <param name="silent" type="select" label="operation mode" help="(Same as 'sed -n', leave at 'normal' unless you know what you're doing)" >
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
34 <option value="">normal</option>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
35 <option value="-n">silent</option>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
36 </param>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
37
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
38 </inputs>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
39 <outputs>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
40 <data format="input" name="output" metadata_source="input" />
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
41 </outputs>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
42 <help>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
43
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
44 **What it does**
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
45
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
46 This tool runs the unix **sed** command on the selected data file.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
47
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
48 .. class:: infomark
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
49
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
50 **TIP:** This tool uses the **extended regular** expression syntax (same as running 'sed -r').
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
51
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
52
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
53
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
54 **Further reading**
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
55
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
56 - Short sed tutorial (http://www.linuxhowtos.org/System/sed_tutorial.htm)
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
57 - Long sed tutorial (http://www.grymoire.com/Unix/Sed.html)
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
58 - sed faq with good examples (http://sed.sourceforge.net/sedfaq.html)
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
59 - sed cheat-sheet (http://www.catonmat.net/download/sed.stream.editor.cheat.sheet.pdf)
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
60 - Collection of useful sed one-liners (http://student.northpark.edu/pemente/sed/sed1line.txt)
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
61
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
62 -----
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
63
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
64 **Sed commands**
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
65
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
66 The most useful sed command is **s** (substitute).
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
67
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
68 **Examples**
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
69
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
70 - **s/hsa//** will remove the first instance of 'hsa' in every line.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
71 - **s/hsa//g** will remove all instances (beacuse of the **g**) of 'hsa' in every line.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
72 - **s/A{4,}/--&amp;--/g** will find sequences of 4 or more consecutive A's, and once found, will surround them with two dashes from each side. The **&amp;** marker is a place holder for 'whatever matched the regular expression'.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
73 - **s/hsa-mir-([^ ]+)/short name: \\1 full name: &amp;/** will find strings such as 'hsa-mir-43a' (the regular expression is 'hsa-mir-' followed by non-space characters) and will replace it will string such as 'short name: 43a full name: hsa-mir-43a'. The **\\1** marker is a place holder for 'whatever matched the first parenthesis' (similar to perl's **$1**) .
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
74
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
75
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
76 **sed's Regular Expression Syntax**
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
77
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
78 The select tool searches the data for lines containing or not containing a match to the given pattern. A Regular Expression is a pattern descibing a certain amount of text.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
79
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
80 - **( ) { } [ ] . * ? + \ ^ $** are all special characters. **\\** can be used to "escape" a special character, allowing that special character to be searched for.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
81 - **^** matches the beginning of a string(but not an internal line).
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
82 - **(** .. **)** groups a particular pattern.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
83 - **{** n or n, or n,m **}** specifies an expected number of repetitions of the preceding pattern.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
84
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
85 - **{n}** The preceding item is matched exactly n times.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
86 - **{n,}** The preceding item ismatched n or more times.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
87 - **{n,m}** The preceding item is matched at least n times but not more than m times.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
88
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
89 - **[** ... **]** creates a character class. Within the brackets, single characters can be placed. A dash (-) may be used to indicate a range such as **a-z**.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
90 - **.** Matches any single character except a newline.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
91 - ***** The preceding item will be matched zero or more times.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
92 - **?** The preceding item is optional and matched at most once.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
93 - **+** The preceding item will be matched one or more times.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
94 - **^** has two meaning:
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
95 - matches the beginning of a line or string.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
96 - indicates negation in a character class. For example, [^...] matches every character except the ones inside brackets.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
97 - **$** matches the end of a line or string.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
98 - **\|** Separates alternate possibilities.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
99
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
100
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
101 **Note**: SED uses extended regular expression syntax, not Perl syntax. **\\d**, **\\w**, **\\s** etc. are **not** supported.
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
102
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
103 </help>
pitagora <yamanaka@genome.rcast.u-tokyo.ac.jp>
parents:
diff changeset
104 </tool>