Commit 93a304d1 authored by rtugade's avatar rtugade
Browse files

Added updated information for genetic algorithm

parent 2e9d7f3d
This diff is collapsed.
......@@ -21,7 +21,7 @@
"metadata": {},
"source": [
"## Abstract\n",
"In performing data analysis, a common task is to search the most appropriate algorithm(s) to best resemble a given system. In this report, we demonstrate the suitability of using a neural network in predicting the potential number of bicycle-sharing users using combined historical rental and weather information. The idea is to augment previous machine learning models and discover the possibility of getting better test accuracy. We used K-Nearest Neighbor, Linear Regression, Ridge Regression, Lasso Regression, Linear Support Vector Machine, Decision Trees, Random Forest, and Gradient Boosting Method as baseline machine learning models. We used a 3 layer fully-connected feed-forward network with 24 hidden nodes. This report shows that such configuration works well the most with 70.6% $r^2$ and 54.9% MAPE. In comparison, RF could predict with 68.0% $r^2$ and 76.1% MAPE while GBM could predict with 67.2% $r^2$ and 69.4% MAPE."
"In performing data analysis, a common task is to search the most appropriate algorithm(s) to best resemble a given system. In this report, we demonstrate the suitability of using a neural network in predicting the potential number of bicycle-sharing users using combined historical rental and weather information. The idea is to augment previous machine learning models and discover the possibility of getting better test accuracy. We used K-Nearest Neighbor, Linear Regression, Ridge Regression, Lasso Regression, Linear Support Vector Machine, Decision Trees, Random Forest, and Gradient Boosting Method as baseline machine learning models. We used a 3 layer fully-connected feed-forward network with 30 hidden nodes. This report shows that such configuration works well the most with 73.2% $r^2$ and 52.7% MAPE. In comparison, RF could predict with 67.9% $r^2$ and 75.9% MAPE while GBM could predict with 67.2% $r^2$ and 68.8% MAPE. "
]
},
{
......@@ -167,13 +167,21 @@
"\n",
"###### Neural Network Modeling\n",
"\n",
"A feed-forward neural network having 56 input nodes, 24 nodes in one hidden layer, and 1 output node was developed. The learning rate from input to hidden and hidden to output were 0.01 and 0.001 respectively. The loss function used is given by:\n",
"A feed-forward neural network having 56 input nodes, 30 nodes in one hidden layer, and 1 output node was developed. The learning rate from input to hidden and hidden to output were both 0.001. The input, hidden, and output activation functions were linear, sigmoid, and sigmoid respectively. These parameters were selected through a genetic algorithm optimizer that aimed to maximize validation accuracy. The ranges of parameter values fed into the genetic algorithm are shown below.\n",
"\n",
"|Parameter|Range of values|\n",
"|:-|:-|\n",
"|hidden nodes|20, 22, 24,...,54, 56|\n",
"|learning rates|0.001, 0.0001|\n",
"|activation functions|sin, relu, tanh, sigmoid|\n",
"\n",
"The loss function being minimized by the neural network is given by:\n",
"\n",
"\\begin{equation}\n",
"\\frac{1}{2}(\\Psi_{NN} - \\Psi_{true})^2\n",
"Error = \\frac{1}{2}(\\Psi_{NN} - \\Psi_{true})^2\n",
"\\end{equation}\n",
"\n",
"where $\\Psi_{NN}$ is the predicted value and $\\Psi_{true}$ is the true value. The input, hidden, and output activation functions were linear, sine, and sigmoid respectively. The neural network was trained and validated using the training and validation sets over 5,000 iterations. The testing set was used to evaluate the predictive accuracy of the model using the coefficient of determination ($r^2$) and mean absolute percentage error (MAPE). The equations are shown below.\n",
"where $\\Psi_{NN}$ is the predicted value and $\\Psi_{true}$ is the true value. The neural network was trained and validated using the training and validation sets over 5,000 iterations. The testing set was used to evaluate the predictive accuracy of the model using the coefficient of determination ($r^2$) and mean absolute percentage error (MAPE). The equations are shown below.\n",
"\n",
"\\begin{equation}\n",
"r^2 = \\frac{\\sum_i(\\hat y_i-\\bar y)}{\\sum_i(y_i-\\bar y)}\n",
......@@ -208,7 +216,7 @@
"|Random forest|max depth|\n",
"|GBM|max depth|\n",
"\n",
"Each model was trained and tested on a range of parameters. The parameters that gave the highest $r^2$ on the test set were identified as the optimal parameters."
"Each model was trained and validated on a range of parameters. The parameters that gave the highest $r^2$ on the validation set were identified as the optimal parameters."
]
},
{
......@@ -222,19 +230,19 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The feed forward neural network was able to predict bike-sharing counts on the test set with 70.6% accuracy, higher than the eight other machine learning models. The MAPE obtained was the lowest at 54.9%. The table below summarizes the predictive accuracies and corresponding parameters of all models evaluated\n",
"The feed forward neural network was able to predict bike-sharing counts on the test set with 73.2% accuracy, higher than the eight other machine learning models. The MAPE obtained was the lowest at 52.7%. The table below summarizes the predictive accuracies and corresponding parameters of all models evaluated\n",
"\n",
"|Model|Parameters|Values|$r^2$ test accuracy|test MAPE|\n",
"|:-|:-|:-|:-|:-|\n",
"|Feed-forward NN|no. of hidden nodes<br>no. of hidden layers<br>activation functions<br>learning rates|24 <br>1 <br>(linear, sine, sigmoid) <br>(0.01, 0.001)|70.6%|54.9%|\n",
"|Random forest|max depth|33|68.0%|76.1%|\n",
"|GBM|max depth|22|67.2%|69.4%|\n",
"|Feed-forward NN|no. of hidden nodes<br>no. of hidden layers<br>activation functions<br>learning rates|30 <br>1 <br>(linear, sigmoid, sigmoid) <br>(0.001, 0.001)|73.2%|52.7%|\n",
"|Random forest|max depth|33|67.9%|75.9%|\n",
"|GBM|max depth|22|67.2%|68.5%|\n",
"|kNN|no. of nearest neighbors|2|62.9%|134.9%|\n",
"|Decision tree|max depth|24|49.5%|79.2%|\n",
"|Lasso regression|alpha|0.0001|48.1%|382.5%|\n",
"|Ridge regression|alpha|10|48.0%|399.4%|\n",
"|Linear regression|-|-|47.1%|424.0%|\n",
"|LSVM|C|1|42.5%|303.6%|"
"|Decision tree|max depth|25|46.0%|75.3%|\n",
"|LSVM|C|10|42.5%|303.6%|"
]
},
{
......@@ -248,9 +256,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We demonstrated that a fully-connected feed-forward neural network could predict hourly bike rentals with the highet $r^2$ and minimum MAPE of 70.6% and 54.9% respectively. The neural network was followed by the forest regressor could predict with 68.0% $r^2$ and 76.1% error. And finally, GBM could predict with 67.2% $r^2$ and 69.4% MAPE.\n",
"We demonstrated that a fully-connected feed-forward neural network could predict hourly bike rentals with the highest accuracy and lowest MAPE. Results showed that using a feed-forward neural network yields a 73.2% $r^2$ and 52.7% MAPE. The neural network was followed by the random forest regressor which could predict with 67.9% $r^2$ and 75.9% MAPE. And finally, GBM with 67.2% $r^2$ and 68.5% MAPE.\n",
"\n",
"Further research can include using recurrent neural networks and ARIMA, which are specifically made for sequential data."
"Further research can include using recurrent neural networks and ARIMA, which are specifically made for sequential data. "
]
},
{
No preview for this file type
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment