Mercurial > repos > iuc > data_manager_mapseq

annotate data_manager_fetch_mapseq_db.py @ 2:d745017a8ea3 draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit abbc840d4383eee1f26abef04b1ce4df1e6e20a2
author iuc
date Fri, 06 Dec 2024 15:43:12 +0000
parents 0453fddf0308
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
1 #!/usr/bin/env python
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
2
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
3 import argparse
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
4 import json
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
5 import os
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
6 import shutil
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
7 import tarfile
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
8 from datetime import datetime
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
9
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
10 import wget
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
11
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
12 DB_paths = {
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
13 "mgnify_v5_lsu": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipeline-5.0/ref-dbs/silva_lsu-20200130.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
14 "mgnify_v5_ssu": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipeline-5.0/ref-dbs/silva_ssu-20200130.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
15 "mgnify_v5_its_unite": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipeline-5.0/ref-dbs/UNITE-20200214.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
16 "mgnify_v5_its_itsonedb": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipeline-5.0/ref-dbs/ITSoneDB-20200214.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
17 "mgnify_v6_lsu": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipelines/tool-dbs/silva-lsu/silva-lsu_138.1.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
18 "mgnify_v6_ssu": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipelines/tool-dbs/silva-ssu/silva-ssu_138.1.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
19 "mgnify_v6_its_unite": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipelines/tool-dbs/unite/unite_9.0.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
20 "mgnify_v6_its_itsonedb": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipelines/tool-dbs/itsonedb/itsonedb_1.141.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
21 "mgnify_v6_pr2": "ftp://ftp.ebi.ac.uk/pub/databases/metagenomics/pipelines/tool-dbs/pr2/pr2_5.0.0.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
22 "test_lsu": "https://zenodo.org/record/8205348/files/test_lsu.tar.gz",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
23 }
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
24
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
25 DB_names = {
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
26 "mgnify_v5_lsu": "MGnify LSU (v5.0.7) - silva_lsu-20200130",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
27 "mgnify_v5_ssu": "MGnify SSU (v5.0.7) - silva_ssu-20200130",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
28 "mgnify_v5_its_unite": "MGnify ITS UNITE (v5.0.7) - UNITE-20200214",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
29 "mgnify_v5_its_itsonedb": "MGnify ITS ITSonedb (v5.0.7) - ITSoneDB-20200214",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
30 "mgnify_v6_lsu": "MGnify LSU (v6.0) - silva_lsu-20240702",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
31 "mgnify_v6_ssu": "MGnify SSU (v6.0) - silva_ssu-20240701",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
32 "mgnify_v6_its_unite": "MGnify ITS UNITE (v6.0) - UNITE-20240702",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
33 "mgnify_v6_its_itsonedb": "MGnify ITS ITSonedb (v6.0) - ITSoneDB-20240702",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
34 "mgnify_v6_pr2": "MGnify PR2 (v6.0) - PR2-20240702",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
35 "test_lsu": "Trimmed LSU Test DB",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
36 }
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
37
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
38
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
39 def download_untar_store(url, tmp_path, dest_path):
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
40 """
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
41 Download a tar.gz file containing one folder,
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
42 extract that folder and move the content inside dest_path
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
43 """
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
44
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
45 extract_path = os.path.join(tmp_path, "extract")
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
46
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
47 os.makedirs(tmp_path, exist_ok=True)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
48
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
49 # download data
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
50 filename = wget.download(url, out=tmp_path)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
51 tarfile_path = os.path.join(tmp_path, filename)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
52 tar = tarfile.open(tarfile_path)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
53 tar.extractall(extract_path)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
54
1
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
55 print(f"Content of folder: {extract_path}", os.listdir(extract_path))
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
56
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
57 # case for mapseq v6: all DB files are directly in the tar.gz file
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
58 # remove the VERSION.txt file since the tool can only handle on .txt file in the DB
0
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
59 if len(list(os.listdir(extract_path))) > 1:
1
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
60 print(f"Found multiple files in {extract_path}. Copy the content.")
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
61 print(f"Copy data to {dest_path}")
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
62 version_file_path = os.path.join(extract_path, "VERSION.txt")
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
63 os.remove(version_file_path)
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
64 shutil.copytree(extract_path, dest_path)
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
65 print("Done !")
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
66
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
67 # case for mapseq v5: all files are in a subfolder in the tar.gz file
0
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
68 else:
1
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
69 print(f"Found a folder in {extract_path}. Copy the content of the folder.")
0
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
70 for folder in os.listdir(extract_path):
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
71 folder_path = os.path.join(extract_path, folder)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
72
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
73 print(f"Copy data to {dest_path}")
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
74 shutil.copytree(folder_path, dest_path)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
75 print("Done !")
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
76
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
77 shutil.rmtree(tmp_path)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
78
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
79
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
80 def main():
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
81 # Parse Command Line
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
82 parser = argparse.ArgumentParser(description="Create data manager JSON.")
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
83 parser.add_argument("--out", dest="output", action="store", help="JSON filename")
1
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
84 parser.add_argument(
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
85 "--version", dest="version", action="store", help="Version of the DB"
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
86 )
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
87 parser.add_argument(
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
88 "--database-type", dest="db_type", action="store", help="Db type"
0453fddf0308 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit b6798b5213eca0d4b176ad3aeef81127dac6dcab
iuc
parents: 0
diff changeset
89 )
0
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
90 parser.add_argument(
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
91 "--test",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
92 action="store_true",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
93 help="option to test the script with an lighted database",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
94 )
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
95
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
96 args = parser.parse_args()
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
97
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
98 # the output file of a DM is a json containing args that can be used by the DM
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
99 # most tools mainly use these args to find the extra_files_path for the DM, which can be used
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
100 # to store the DB data
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
101 with open(args.output) as fh:
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
102 params = json.load(fh)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
103
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
104 workdir = params["output_data"][0]["extra_files_path"]
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
105 os.mkdir(workdir)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
106
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
107 time = datetime.utcnow().strftime("%Y-%m-%d")
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
108 db_value = f"{args.db_type}_from_{time}"
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
109
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
110 # output paths
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
111 db_path = os.path.join(workdir, db_value)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
112 tmp_path = os.path.join(workdir, "tmp")
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
113
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
114 # create DB
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
115 if args.test:
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
116 url = DB_paths["test_lsu"]
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
117 else:
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
118 url = DB_paths[args.db_type]
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
119
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
120 # download data
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
121 download_untar_store(url, tmp_path, db_path)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
122
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
123 db_name = DB_names[args.db_type]
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
124 # Update Data Manager JSON and write to file
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
125 data_manager_entry = {
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
126 "data_tables": {
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
127 "mapseq_db": {
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
128 "value": db_value,
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
129 "name": f"{db_name} downloaded at {time}",
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
130 "version": args.version,
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
131 "path": db_path,
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
132 }
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
133 }
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
134 }
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
135
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
136 with open(os.path.join(args.output), "w+") as fh:
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
137 json.dump(data_manager_entry, fh, sort_keys=True)
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
138
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
139
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
140 if __name__ == "__main__":
3f7f3ef77c34 planemo upload for repository https://github.com/galaxyproject/tools-iuc/ commit 960c1f69ec78943dfe72eff0a0992a2ce15d2555
iuc
parents:
diff changeset
141 main()