diff docs/scripts/txt/AnalyzeTextFilesData.txt @ 0:4816e4a8ae95 draft default tip

Uploaded

--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/docs/scripts/txt/AnalyzeTextFilesData.txt	Wed Jan 20 09:23:18 2016 -0500
@@ -0,0 +1,283 @@
+NAME
+    AnalyzeTextFilesData.pl - Analyze numerical coulmn data in TextFile(s)
+
+SYNOPSIS
+    AnalyzeTextFilesData.pl TextFile(s)...
+
+    AnalyzeTextFilesData.pl [-c, --colmode colnum | collabel] [--columns
+    "colnum,[colnum,...]" | "collabel,[collabel,...]" | All] [--columnpairs
+    "colnum,colnum,[colnum,colnum]..." |
+    "collabel,collabel,[collabel,collabel]..." | AllPairs] [-d, --detail
+    infolevel] [-f, --fast] [--frequencybins number |
+    "number,number,[number,...]"] [-h, --help] [--indelim comma | semicolon]
+    [--klargest number] [--ksmallest number] [-m, --mode
+    DescriptiveStatisticsBasic | DescriptiveStatisticsAll | All |
+    "function1, [function2,...]"] [-o, --overwrite] [--outdelim comma | tab
+    | semicolon] [-p, --precision number] [-q, --quote yes | no] [-r, --root
+    rootname] [--trimfraction number] [-w, --workingdir dirname]
+    TextFiles(s)...
+
+DESCRIPTION
+    Anaylze numerical column data in *TextFile(s)* using a combination of
+    various statistical functions; Non-numerical values are simply ignored.
+    For *Correlation, RSquare, and Covariance* analysis, the count of valid
+    values in specifed column pair must be same; otherwise, column pair is
+    ignored. The file names are separated by space. The valid file
+    extensions are *.csv* and *.tsv* for comma/semicolon and tab delimited
+    text files respectively. All other file names are ignored. All the text
+    files in a current directory can be specified by **.csv*, **.tsv*, or
+    the current directory name. The --indelim option determines the format
+    of *TextFile(s)*. Any file which doesn't correspond to the format
+    indicated by --indelim option is ignored.
+
+OPTIONS
+    -c, --colmode *colnum | collabel*
+        Specify how columns are identified in TextFile(s): using column
+        number or column label. Possible values: *colnum or collabel*.
+        Default value: *colnum*.
+
+    --columns *"colnum,[colnum,...]" | "collabel,[collabel]..." | All*
+        This value is mode specific. It's a list of comma delimited columns
+        to use for data analysis. Default value: *First column*.
+
+        This value is ignored during *Correlation/Pearson Correlation* and
+        *Covariance* data analysis; -coulmnparis option is used instead.
+
+        For *colnum* value of -c, --colmode option, input values format is:
+        *colnum,colnum,...*. Example:
+
+           1,3,5
+
+        For *collabel* value of -c, --colmode option, input values format
+        is: *collabel,collabel,..*. Example:
+
+            ALogP,MolWeight,EC50
+
+    --columnpairs *"colnum,colnum,[colnum,colnum,...]" |
+    "collabel,collabel,[collabel,collabel,...]" | AllPairs*
+        This value is mode specific and is only used for *Correlation,
+        PearsonCorrelation, or Covariance* value of -m, --mode option. It is
+        a comma delimited list of column pairs to use for data analysis
+        during *Correlation* and *Covariance* calculations. Default value:
+        *First column, Second column*.
+
+        For *colnum* value of -c, --colmode option, input values format is:
+        *colnum,colnum,[colnum,colnum]...*. Example:
+
+            1,3,5,6,1,6
+
+        For *collabel* value of -c, --colmode option, input values format
+        is: *collabel,collabel,[collabel,collabel]..*. Example:
+
+            MolWeight,EC50,NumN+O,PSA
+
+        For *AllPairs* value of --columnparis option, all column pairs are
+        used for *Correlation* and *Covariance* calculations.
+
+    -d, --detail *infolevel*
+        Level of information to print about column values being ignored.
+        Default: *1*. Possible values: 1, 2, 3, or 4.
+
+    -f, --fast
+        In this mode, all the columns specified for analysis are assumed to
+        contain numerical data and no checking is performed before analysis.
+        By default, only numerical data is used for analysis; other types of
+        column data is ignored.
+
+    --frequencybins *number | "number,number,[number,...]"*
+        Specify number of bins or bin range to use for frequency analysis.
+        Default value: *10*
+
+        Number of bins value along with the smallest and largest value for a
+        column is used to group the column values into different groups.
+
+        The bin range list is used to group values for a column into
+        different groups; It must contain values in ascending order.
+        Examples:
+
+            10,20,30
+            0.1,0.2,0.3,0.4,0.5
+
+        The frequency value calculated for a specific bin corresponds to all
+        the column values which are greater than the previous bin value and
+        less than or equal to the current bin value.
+
+    -h, --help
+        Print this help message.
+
+    --indelim *comma | semicolon*
+        Input delimiter for CSV *TextFile(s)*. Possible values: *comma or
+        semicolon*. Default value: *comma*. For TSV files, this option is
+        ignored and *tab* is used as a delimiter.
+
+    --klargest *number*
+        Kth largest value to find by *KLargest* function. Default value: *2*
+        Valid values: positive integers.
+
+    --ksmallest *number*
+        Kth smallest value to find by *KSmallest* function. Default value:
+        *2*. Valid values: positive integers.
+
+    -m, --mode *DescriptiveStatisticsBasic | DescriptiveStatisticsAll | All
+    | "function1, [function2,...]"*
+        Specify how to analyze data in TextFile(s): calculate basic or all
+        descriptive statistics; or use a comma delimited list of supported
+        statistical functions. Possible values: *DescriptiveStatisticsBasic
+        | DescriptiveStatisticsAll | "function1,[function2]..."*. Default
+        value: *DescriptiveStatisticsBasic*
+
+        *DescriptiveStatisticsBasic* includes these functions: *Count,
+        Maximum, Minimum, Mean, Median, Sum, StandardDeviation,
+        StandardError, Variance*.
+
+        *DescriptiveStatisticsAll*, in addition to
+        *DescriptiveStatisticsBasic* functions, includes: *GeometricMean,
+        Frequency, HarmonicMean, KLargest, KSmallest, Kurtosis, Mode,
+        RSquare, Skewness, TrimMean*.
+
+        *All* uses complete list of supported functions: *Average,
+        AverageDeviation, Correlation, Count, Covariance, GeometricMean,
+        Frequency, HarmonicMean, KLargest, KSmallest, Kurtosis, Maximum,
+        Minimum, Mean, Median, Mode, RSquare, Skewness, Sum, SumOfSquares,
+        StandardDeviation, StandardDeviationN, StandardError,
+        StandardScores, StandardScoresN, TrimMean, Variance, VarianceN*. The
+        function names ending with N calculate corresponding values assuming
+        an entire population instead of a population sample.
+
+        Here are the formulas for these functions:
+
+        Average: See Mean
+
+        AverageDeviation: SUM( ABS(x[i] - Xmean) ) / n
+
+        Correlation: See Pearson Correlation
+
+        Covariance: SUM( (x[i] - Xmean)(y[i] - Ymean) ) / n
+
+        GeometricMean: NthROOT( PRODUCT(x[i]) )
+
+        HarmonicMean: 1 / ( SUM(1/x[i]) / n )
+
+        Mean: SUM( x[i] ) / n
+
+        Median: Xsorted[(n - 1)/2 + 1] for even values of n; (Xsorted[n/2] +
+        Xsorted[n/2 + 1])/2 for odd values of n.
+
+        Kurtosis: [ {n(n + 1)/(n - 1)(n - 2)(n - 3)} SUM{ ((x[i] -
+        Xmean)/STDDEV)^4 } ] - {3((n - 1)^2)}/{(n - 2)(n-3)}
+
+        PearsonCorrelation: SUM( (x[i] - Xmean)(y[i] - Ymean) ) / SQRT( SUM(
+        (x[i] - Xmean)^2 ) (SUM( (y[i] - Ymean)^2 )) )
+
+        RSquare: PearsonCorrelation^2
+
+        Skewness: {n/(n - 1)(n - 2)} SUM{ ((x[i] - Xmean)/STDDEV)^3 }
+
+        StandardDeviation: SQRT ( SUM( (x[i] - Mean)^2 ) / (n - 1) )
+
+        StandardDeviationN: SQRT ( SUM( (x[i] - Mean)^2 ) / n )
+
+        StandardError: StandardDeviation / SQRT( n )
+
+        StandardScore: (x[i] - Mean) / (n - 1)
+
+        StandardScoreN: (x[i] - Mean) / n
+
+        Variance: SUM( (x[i] - Xmean)^2 / (n - 1) )
+
+        VarianceN: SUM( (x[i] - Xmean)^2 / n )
+
+    -o, --overwrite
+        Overwrite existing files.
+
+    --outdelim *comma | tab | semicolon*
+        Output text file delimiter. Possible values: *comma, tab, or
+        semicolon* Default value: *comma*.
+
+    -p, --precision *number*
+        Precision of calculated values in the output file. Default: up to
+        *2* decimal places. Valid values: positive integers.
+
+    -q, --quote *yes | no*
+        Put quotes around column values in output text file. Possible
+        values: *yes or no*. Default value: *yes*.
+
+    -r, --root *rootname*
+        New text file name is generated using the root: <Root>.<Ext>.
+        Default new file name: <InitialTextFileName><Mode>.<Ext>. Based on
+        the specified analysis, <Mode> corresponds to one of these values:
+        DescriptiveStatisticsBasic, DescriptiveStatisticsAll, AllStatistics,
+        SpecifiedStatistics, Covariance, Correlation, Frequency, or
+        StandardScores. The csv, and tsv <Ext> values are used for
+        comma/semicolon, and tab delimited text files respectively. This
+        option is ignored for multiple input files.
+
+    --trimfraction *number*
+        Fraction of data to exclude from the top and bottom of the data set
+        during *TrimMean* calculation. Default value: *0.1*. Valid values: >
+        0 and < 1.
+
+    -w --workingdir *text*
+        Location of working directory. Default: current directory.
+
+EXAMPLES
+    To calculate basic statistics for data in first column and generate a
+    NewSample1DescriptiveStatisticsBasic.csv file, type:
+
+        % AnalyzeTextFilesData.pl -o -r NewSample1 Sample1.csv
+
+    To calculate basic statistics for data in third column and generate a
+    NewSample1DescriptiveStatisticsBasic.csv file, type:
+
+        % AnalyzeTextFilesData.pl --columns 3 -o -r NewSample1 Sample1.csv
+
+    To calculate basic statistics for data in MolWeight column and generate
+    a NewSample1DescriptiveStatisticsBasic.csv file, type:
+
+        % AnalyzeTextFilesData.pl -colmode collabel --columns MolWeight -o
+        -r NewSample1 Sample1.csv
+
+    To calculate all available statistics for data in third column and all
+    column pairs, and generate NewSample1DescriptiveStatisticsAll.csv,
+    NewSample1CorrelationMatrix.csv, NewSample1CorrelationMatrix.csv, and
+    NewSample1MolWeightFrequencyAnalysis.csv files, type:
+
+        % AnalyzeTextFilesData.pl -m DescriptiveStatisticsAll --columns 3 -o
+        --columnpairs AllPairs -r NewSample1 Sample1.csv
+
+    To compute frequency distribution of data in third column into five bins
+    and generate NewSample1MolWeightFrequencyAnalysis.csv, type:
+
+        % AnalyzeTextFilesData.pl -m Frequency --frequencybins 5 --columns 3
+        -o -r NewSample1 Sample1.csv
+
+    To compute frequency distribution of data in third column into specified
+    bin range values, and generate NewSample1MolWeightFrequencyAnalysis.csv,
+    type:
+
+        % AnalyzeTextFilesData.pl -m Frequency --frequencybins "100,200,400"
+        --columns 3 -o -r NewSample1 Sample1.csv
+
+    To calculate all available statistics for data in all columns and column
+    pairs, type:
+
+        % AnalyzeTextFilesData.pl -m All --columns  All --columnpairs
+        AllPairs -o -r NewSample1 Sample1.csv
+
+AUTHOR
+    Manish Sud <[e-mail]>
+
+SEE ALSO
+    JoinTextFiles.pl, MergeTextFilesWithSD.pl, ModifyTextFilesFormat.pl,
+    SplitTextFiles.pl, TextFilesToHTML.pl
+
+COPYRIGHT
+    Copyright (C) 2015 Manish Sud. All rights reserved.
+
+    This file is part of MayaChemTools.
+
+    MayaChemTools is free software; you can redistribute it and/or modify it
+    under the terms of the GNU Lesser General Public License as published by
+    the Free Software Foundation; either version 3 of the License, or (at
+    your option) any later version.
+