View source for CSV Statistical Analysis using wolframscript


== CSV Statistical Analysis in Wolfram Mathematica ==

'''CSV Statistical Analysis in Wolfram Mathematica''' is a comprehensive computational methodology for performing statistical operations on comma-separated values (CSV) data files using Wolfram Mathematica. This approach demonstrates a complete statistical workflow that processes CSV data through multiple analytical stages, from basic descriptive statistics to advanced regression modeling, hypothesis testing, and residual analysis.

== Overview ==

The analysis utilizes Mathematica's built-in statistical functions and visualization capabilities to provide comprehensive insights into dataset relationships and patterns. This implementation specifically uses the '''Wolfram Engine kernel''' within the '''ZCubes (Z³) Application Platform''' environment, demonstrating the versatility of Wolfram Language across different computational platforms. The methodology combines data import capabilities with advanced statistical functions to deliver descriptive statistics, correlation analysis, regression modeling, and diagnostic visualizations for a five-variable dataset.

== Installation and Setup ==

=== Wolfram Engine and ZCubes Integration ===

To replicate this analysis using the ZCubes platform with Wolfram integration, follow the comprehensive installation guide available at the [https://wiki.zcubes.com/Z%5E3_Mathematica_Integration Z³ Mathematica Integration] documentation.

'''Key Installation Steps:'''
# Install Wolfram Desktop or Mathematica from the Wolfram User Portal
# Install Anaconda or Jupyter Notebook for kernel management
# Download and install the Wolfram Jupyter Kernel from the official GitHub repository
# Configure the Wolfram kernel using the command: `ZADDSERVERLANGUAGE("wolframscript", "wolframlanguage14.2")`
# Configure API and Jupyter settings in ZCubes Advanced Settings panel
# Verify installation by executing Wolfram Language code within ZCubes

=== ZCubes Environment Configuration ===

This analysis was conducted within the '''ZCubes (Z³) Application Platform''' environment, which provides an integrated development environment for mathematical and statistical computing. The ZCubes platform supports Wolfram Language kernel integration through its Omniglot capability, enabling seamless execution of Mathematica code within a web-based interface alongside other programming languages.

== Dataset Description ==

[[File:Csv 2.png|thumb]]

The analysis employs a synthetic dataset specifically designed for demonstrating statistical modeling techniques.[2] The dataset contains **12 observations** across **five variables**:

* '''X1''': Primary predictor variable ranging from 27.41 to 74.77
* '''X2''': Secondary predictor variable ranging from 15.05 to 35.75  
* '''X3''': Tertiary predictor variable ranging from 2.79 to 18.84
* '''Noise''': Random noise component ranging from -4.59 to 6.58
* '''Y''': Dependent variable ranging from 99.47 to 213.66

The dataset demonstrates a structured relationship where Y is influenced by the three predictor variables X1, X2, and X3, with an additional noise component to simulate real-world data variability. This synthetic nature allows for controlled statistical analysis while maintaining realistic data characteristics suitable for regression modeling and hypothesis testing.

== Statistical Results ==

=== Descriptive Statistics ===

Comprehensive descriptive statistics reveal the central tendencies and variability within each variable:

* '''X1''': Mean = 45.399, Standard Deviation = 18.6360
* '''X2''': Mean = 26.565, Standard Deviation = 7.1559  
* '''X3''': Mean = 9.46336, Standard Deviation = 5.75943
* '''Y''': Mean = 160.211, Standard Deviation = 47.4838

The descriptive statistics indicate substantial variability in X1 (coefficient of variation = 41.1%) and Y (coefficient of variation = 29.6%), while X2 and X3 show more moderate variability. This variation provides sufficient statistical power for meaningful correlation and regression analyses.

=== Correlation Analysis ===

Pairwise correlation analysis reveals distinct relationships between predictor and response variables:

* '''X1 vs Y''': 0.956843 (strong positive correlation)
* '''X2 vs Y''': 0.192992 (weak positive correlation)
* '''X3 vs Y''': -0.428338 (moderate negative correlation)

The correlation matrix demonstrates that X1 serves as the primary driver of Y variation, explaining over 91% of the shared variance. X3 shows a moderate inverse relationship, while X2 exhibits minimal correlation with the response variable.

=== Linear Regression ===

Simple linear regression between X1 and Y demonstrates exceptional model fit:

* '''R-squared''': 0.915548 (91.6% variance explained)
* '''Regression Equation''': <math>Y = 49.533 + 2.4379x</math>

The linear model indicates that for each unit increase in X1, Y increases by approximately 2.44 units, with a baseline intercept of 49.53 when X1 equals zero.

=== Multiple Regression Analysis ===

The comprehensive multiple regression model incorporating all three predictor variables demonstrates exceptional predictive capability:

* '''Multiple R-squared''': 0.995941 (99.6% variance explained)
* '''Adjusted R-squared''': 0.994201 (99.4% adjusted for model complexity)
* '''Multiple Regression Model''': <math>Y = -2.57948 + 2.4929x_1 + 1.89864x_2 - 0.0868901x_3</math>

The multiple regression model reveals that:
- X1 contributes positively with a coefficient of 2.49
- X2 provides additional positive contribution (1.90)
- X3 has a small negative effect (-0.087)

The extremely high R-squared values indicate that the three-variable model captures virtually all systematic variation in the response variable.

=== Hypothesis Testing ===

Statistical significance testing confirms model validity using both automated and manual calculation approaches:

* '''Automated T-test result''': 0.00806328
* '''Manual Welch's T-test calculation''':
** '''T-statistic''': 3.129
** '''Degrees of freedom''': 12.8859
** '''P-value''': 0.00806328

The extremely low p-value (< 0.01) provides strong evidence against the null hypothesis of no difference between X1 and X2 means, indicating statistical significance at the 99% confidence level using Welch's t-test methodology for unequal variances.

=== Residual Analysis ===

Comprehensive residual analysis validates fundamental regression assumptions:

* '''Residual mean''': 1.80865 × 10<sup>−14</sup> (effectively zero, confirming unbiased estimates)
* '''Residual standard deviation''': 3.02535 (indicating model precision)

The near-zero residual mean confirms that the model provides unbiased predictions, while the relatively small residual standard deviation (compared to Y's standard deviation of 47.48) demonstrates excellent model fit.

== Results Interpretation ==
[[File:Result 1.png|thumb]]
[[File:Result 2.png|thumb]]
[[File:Result 3.png|thumb]]
[[File:Result 4.png|thumb]]
The statistical analysis yields several key findings that demonstrate the effectiveness of the multiple regression approach:

**Model Performance**: The multiple regression model achieves exceptional predictive accuracy, explaining 99.6% of the variance in the dependent variable Y. This performance substantially exceeds the simple linear regression model (91.6% variance explained), highlighting the value of incorporating multiple predictors.

**Variable Importance**: X1 emerges as the dominant predictor with the strongest correlation (0.957) and largest regression coefficient (2.49). X2 provides meaningful additional predictive power despite its weak individual correlation with Y, demonstrating the importance of multivariate analysis. X3 contributes a small but statistically meaningful negative effect.

**Statistical Validity**: The combination of high R-squared values, statistically significant hypothesis tests (p < 0.01), and validated residual patterns confirms the robustness and reliability of the statistical model for both explanatory and predictive applications.

**Practical Implications**: The model's exceptional fit suggests strong underlying relationships between the predictor variables and the response, making it suitable for forecasting and decision-making applications where accurate prediction of Y values is required.

== Code Implementation ==

The complete Mathematica implementation utilizes structured programming approaches within the ZCubes environment:

<syntaxhighlight lang="mathematica">
(* Statistical Analysis Script - ZCubes Z³ Environment Implementation *)

(* Read the CSV file *)
csvData = Import["D:/ZAP_Testing/synthetic_data.csv", "CSV"];

(* Extract header and data rows *)
header = csvData[[1]];
dataRows = csvData[[2;;]];

(* Extract individual columns and ensure they are numeric *)
x1Data = N[dataRows[[All, 1]]];
x2Data = N[dataRows[[All, 2]]];
x3Data = N[dataRows[[All, 3]]];
noiseData = N[dataRows[[All, 4]]];
yData = N[dataRows[[All, 5]]];

(* DESCRIPTIVE STATISTICS *)
Print["=== DESCRIPTIVE STATISTICS ==="];
Print["X1 - Mean: ", Mean[x1Data], ", Std: ", StandardDeviation[x1Data]];
Print["X2 - Mean: ", Mean[x2Data], ", Std: ", StandardDeviation[x2Data]];
Print["X3 - Mean: ", Mean[x3Data], ", Std: ", StandardDeviation[x3Data]];
Print["Y - Mean: ", Mean[yData], ", Std: ", StandardDeviation[yData]];

(* CORRELATION ANALYSIS *)
Print["\n=== CORRELATIONS ==="];
Print["X1 vs Y: ", Correlation[x1Data, yData]];
Print["X2 vs Y: ", Correlation[x2Data, yData]];
Print["X3 vs Y: ", Correlation[x3Data, yData]];

(* LINEAR REGRESSION *)
Print["\n=== LINEAR REGRESSION ==="];
model = LinearModelFit[Transpose[{x1Data, yData}], x, x];
Print["R-squared: ", model["RSquared"]];
Print["Equation: Y = ", model["BestFit"]];

(* MULTIPLE REGRESSION *)
Print["\n=== MULTIPLE REGRESSION ==="];
multiModel = LinearModelFit[Transpose[{x1Data, x2Data, x3Data, yData}], {x1, x2, x3}, {x1, x2, x3}];
Print["Multiple R-squared: ", multiModel["RSquared"]];
Print["Adjusted R-squared: ", multiModel["AdjustedRSquared"]];
Print["Model: Y = ", multiModel["BestFit"]];

(* HYPOTHESIS TESTING - FIXED *)
Print["\n=== HYPOTHESIS TESTING ==="];
tTestResult = TTest[{x1Data, x2Data}];
Print["T-test result: ", tTestResult];

(* Manual T-test calculation using Welch's method *)
n1 = Length[x1Data]; n2 = Length[x2Data];
mean1 = Mean[x1Data]; mean2 = Mean[x2Data];
var1 = Variance[x1Data]; var2 = Variance[x2Data];
tStat = (mean1 - mean2)/Sqrt[var1/n1 + var2/n2];
df = (var1/n1 + var2/n2)^2/((var1/n1)^2/(n1-1) + (var2/n2)^2/(n2-1));
pValue = 2*(1 - CDF[StudentTDistribution[df], Abs[tStat]]);

Print["Manual T-test calculation:"];
Print["T-statistic: ", tStat];
Print["Degrees of freedom: ", df];
Print["P-value: ", pValue];

(* RESIDUAL ANALYSIS *)
Print["\n=== RESIDUAL ANALYSIS ==="];
residuals = multiModel["FitResiduals"];
Print["Residual mean: ", Mean[residuals]];
Print["Residual std: ", StandardDeviation[residuals]];

(* VISUALIZATION *)
Print["\n=== CREATING PLOTS ==="];
plot1 = ListPlot[Transpose[{x1Data, yData}], PlotLabel -> "X1 vs Y", PlotStyle -> Blue];
plot2 = Histogram[yData, PlotLabel -> "Y Distribution"];
plot3 = ListPlot[Transpose[{multiModel["PredictedResponse"], residuals}], 
  PlotLabel -> "Residuals vs Fitted", PlotStyle -> Red];

GraphicsGrid[{{plot1, plot2}, {plot3, ""}}]

Print["\n=== ANALYSIS COMPLETE ==="];
</syntaxhighlight>

== Technical Features ==

=== ZCubes Z³ Environment Integration ===

The implementation leverages the ZCubes (Z³) Application Platform environment, which provides:

* '''Omniglot Capability''': Multi-language programming environment supporting Wolfram Language integration
* '''Web-based Interface''': Browser-accessible computational environment with collaborative features
* '''Wolfram Kernel Integration''': Native support for Wolfram Language execution through Jupyter kernel management
* '''Cross-platform Compatibility''': Operating system independent deployment with seamless language switching

== Statistical Significance ==

The analysis demonstrates exceptional predictive capability with the multiple regression model explaining approximately '''99.6%''' of the variance in the dependent variable. The combination of high R-squared values (0.995941), extremely low p-values (0.00806328), and validated residual patterns indicates a robust statistical model suitable for both explanatory and predictive purposes across diverse analytical applications.

== See also ==

* [[Z^3 Mathematica Integration]]
* [[Z functions with CSV and XLSX files]]
* [[Manuals/calci/LINEST|LINEST (YRange,XRanges,Constant,Stats)]]

== External links ==

* [https://www.wolfram.com/mathematica/ Wolfram Mathematica Official Website]
* [https://www.wolfram.com/engine/ Wolfram Engine Free Download]
* [https://wiki.zcubes.com/Z%5E3_Mathematica_Integration Z³ Mathematica Integration - Installation Guide]
* [https://reference.wolfram.com/language/guide/Statistics.html Wolfram Language Statistical Documentation]
* [https://reference.wolfram.com/language/ref/LinearModelFit.html LinearModelFit Documentation]
* [https://github.com/WolframResearch/WolframLanguageForJupyter Wolfram Language for Jupyter GitHub Repository]

[[Category:Statistical software]]
[[Category:Data analysis]]
[[Category:Wolfram Mathematica]]
[[Category:Wolfram Engine]]
[[Category:Regression analysis]]
[[Category:Data visualization]]
[[Category:Statistical computing]]
[[Category:CSV data processing]]
[[Category:Hypothesis testing]]
[[Category:ZCubes platform]]