Mercurial > repos > stevecassidy > nltktools

annotate g_frequency.py @ 3:0df72a8ab095 draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
author stevecassidy
date Mon, 20 Nov 2017 22:52:11 -0500
parents a47980ef2b96
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
1 import nltk
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
2 from nltk import FreqDist
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
3 import argparse
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
4
2
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
5 nltk.download('punkt', quiet=True)
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
6
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
7
0
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
8 def arguments():
2
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
9 parser = argparse.ArgumentParser(description="generate a word frequency table from a text")
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
10 parser.add_argument('--input', required=True, action="store", type=str, help="input text file")
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
11 parser.add_argument('--output', required=True, action="store", type=str, help="output file path")
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
12 return parser.parse_args()
0
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
13
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
14
3
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
15 def frequency(textfiles, out_file):
0
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
16 """Input: a text file
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
17 Output: a table of word frequency with three columns for Word, Count and Percent frequency
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
18 """
1
fb617586f4b2 planemo upload commit a81826fe44f09a3710a35c183aa88b745aeec064-dirty
stevecassidy
parents: 0
diff changeset
19
3
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
20 words = []
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
21 for textfile in textfiles:
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
22 with open(textfile, 'r') as fd:
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
23 text = fd.read()
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
24
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
25 words.extend(nltk.word_tokenize(text))
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
26
2
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
27 fdist = FreqDist(words)
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
28 total = float(fdist.N())
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
29
1
fb617586f4b2 planemo upload commit a81826fe44f09a3710a35c183aa88b745aeec064-dirty
stevecassidy
parents: 0
diff changeset
30 with open(out_file, 'w') as output:
fb617586f4b2 planemo upload commit a81826fe44f09a3710a35c183aa88b745aeec064-dirty
stevecassidy
parents: 0
diff changeset
31 output.write("Word\tCount\tPercent\n")
3
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
32 for pair in sorted(fdist.items()):
2
a47980ef2b96 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit b5b26e9118f2ad8af109d606746b39a5588f0511-dirty
stevecassidy
parents: 1
diff changeset
33 output.write("{pair[0]}\t{pair[1]}\t{pc:.2f}\n".format(pair=pair, pc=100 * pair[1] / total))
0
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
34
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
35
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
36 if __name__ == '__main__':
e991d4e60c17 planemo upload commit 0203cb3a0b40d9348674b2b098af805e2986abca-dirty
stevecassidy
parents:
diff changeset
37 args = arguments()
3
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
38 textfiles = args.input.split(',')
0df72a8ab095 planemo upload for repository https://github.com/Alveo/alveo-galaxy-tools commit f2432aaedd36ae7662873623d8861d0982dffdd2
stevecassidy
parents: 2
diff changeset
39 frequency(textfiles, args.output)