{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Setup Libraries and Environment\n\nWe'll start with loading the different libraries that we are likely to use.\n\nIn this case, the following libraries are called:<br />\n1.)  numpy  -  This library will give us mathematical processes<br />\n2.)  pandas - This library will allow us to manage the data<br />\n3.)  sklearn - This library will provide us with the machine learning models<br /> \n4.)  seaborn -  This library will help with investigation of features and exploration of data<br />\n5.)  matplotlib.pyplot  -  This will give us the plotting capabilities from the matplot library\n\nFurthermore, we will check the availability of the data and report the file-paths for the data in this project.","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sns\n#import sklearn as ...  #Which methods from the SK library will be determined later.\n\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-06-05T05:40:28.128969Z","iopub.execute_input":"2022-06-05T05:40:28.129421Z","iopub.status.idle":"2022-06-05T05:40:28.142828Z","shell.execute_reply.started":"2022-06-05T05:40:28.129386Z","shell.execute_reply":"2022-06-05T05:40:28.141788Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#  Read Datasets\n\nNext up, we will read in the datasets, and verify that they have been read in correctly by<br\\>\nprinting the first 5 rows of both the training data and the test data.","metadata":{}},{"cell_type":"code","source":"train_data_filepath = \"/kaggle/input/challenge-datascience-wind-turbine-cs/training_set.csv\" #store the file-path for the training data\ntest_data_filepath = \"/kaggle/input/challenge-datascience-wind-turbine-cs/test_set.csv\" #store the file-path for the test data\n\n# Uncomment when first running or to reload original data\ntraining_data = pd.read_csv(train_data_filepath)  #read the training data in \n\n#report the first few rows of the data and shape of the data frame.\n#training_data.shape\ntraining_data.head() #print the first 5 rows of the training data","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:28.236355Z","iopub.execute_input":"2022-06-05T05:40:28.236762Z","iopub.status.idle":"2022-06-05T05:40:35.681621Z","shell.execute_reply.started":"2022-06-05T05:40:28.236704Z","shell.execute_reply":"2022-06-05T05:40:35.68052Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#test_data = pd.read_csv(test_data_filepath)\n#test_data.head()\n#test_data.shape","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:35.683571Z","iopub.execute_input":"2022-06-05T05:40:35.684528Z","iopub.status.idle":"2022-06-05T05:40:35.689556Z","shell.execute_reply.started":"2022-06-05T05:40:35.68448Z","shell.execute_reply":"2022-06-05T05:40:35.688311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Summary, Visualization, and Data Cleaning\n\nSince the data has now been read in, we can now investigate the training set.\n","metadata":{}},{"cell_type":"markdown","source":"## Summary\n\n### Response Variable\nWe begin with investiagting the response variable of \"TARGET\"","metadata":{}},{"cell_type":"code","source":"training_data.TARGET.describe() #Getting a picture of the summary statistics for this data.","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:35.690898Z","iopub.execute_input":"2022-06-05T05:40:35.691917Z","iopub.status.idle":"2022-06-05T05:40:35.728319Z","shell.execute_reply.started":"2022-06-05T05:40:35.69188Z","shell.execute_reply":"2022-06-05T05:40:35.727154Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"training_data.TARGET.isna().sum() #Find out how many NA values are in the response variable.","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:35.732686Z","iopub.execute_input":"2022-06-05T05:40:35.733056Z","iopub.status.idle":"2022-06-05T05:40:35.740143Z","shell.execute_reply.started":"2022-06-05T05:40:35.733025Z","shell.execute_reply":"2022-06-05T05:40:35.739255Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.displot(training_data.TARGET)","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:35.741278Z","iopub.execute_input":"2022-06-05T05:40:35.741785Z","iopub.status.idle":"2022-06-05T05:40:36.822381Z","shell.execute_reply.started":"2022-06-05T05:40:35.741747Z","shell.execute_reply":"2022-06-05T05:40:36.821401Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Well, this is not great, but it is interesting.  According to the data, a significant number of datapoints in the TARGET are actually... negative with a maximum negative value of 19 KW?  This is somewhat weird because, unless the sensors are drawing 19kW (around 25 horsepower), something is going very weird here.  \nIf I had to guess, this is noise in the data surrounding the power production when the wind speed is below cut-in velocity (typically 6 mph or so.)\n\nRegardless, this is then 0 inflated data, so we need to investigate the other features to see if there is a distinguishing characteristic in the sensors that can indicate whether the turbine is producing energy or not.\n\nThis seems to be a case where we will need a two stage model (hurdle model.)  In the first stage, we will need a classifier of whether or not there is any production.  From there, we will build one model for the \"no production\" (the average isn't \"0\", so we'll still need to predict, even if the values are negative.  We will also need to build a model for where the production is positive. \n\nTo start with, we need to identify a cutoff for an indicator value.  Since there are some \"small\" non-zero production values, we need to focus in on the values near zero (say, from 0 to 10kW?)","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots()\nsns.histplot(training_data.TARGET, ax=ax)\nax.set_xlim([-1,200])\n","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:36.823529Z","iopub.execute_input":"2022-06-05T05:40:36.823951Z","iopub.status.idle":"2022-06-05T05:40:37.667779Z","shell.execute_reply.started":"2022-06-05T05:40:36.823918Z","shell.execute_reply":"2022-06-05T05:40:37.665796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots()\nsns.histplot(training_data.TARGET, ax=ax, binwidth = 0.25)\nax.set_xlim([-1,2])","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:37.668954Z","iopub.execute_input":"2022-06-05T05:40:37.669261Z","iopub.status.idle":"2022-06-05T05:40:54.778562Z","shell.execute_reply.started":"2022-06-05T05:40:37.669233Z","shell.execute_reply":"2022-06-05T05:40:54.777603Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Creating an indicator variable.\n\nLooks like our cutoff value is around 0.3kW.  \n\n","metadata":{}},{"cell_type":"code","source":"training_data['Indicator'] = np.where(training_data.TARGET < 0.3, \"Low\", \"High\")\ntraining_data.Indicator.describe()","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:54.779898Z","iopub.execute_input":"2022-06-05T05:40:54.78091Z","iopub.status.idle":"2022-06-05T05:40:54.95669Z","shell.execute_reply.started":"2022-06-05T05:40:54.780864Z","shell.execute_reply":"2022-06-05T05:40:54.955957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"###  Potential Features and Models for stage 1:  Binary Classifier\n\nOkay, we've gone ahead and created a binary variable for classifying our output as \"Low\" or \"High\".  Now, we need to figure out how to use our other potential variables to classify into one of these two categories.  There are LOTS of options:\n\n* Logistic Regression\n* Probit Models\n* Decision Trees\n* Random Forests\n* Baysian network\n* SVM\n* Neural Networks\n* Others?\n\nAt this point, we need to do two things before we build our model to classify.  First, we need to investigate the other columns and find out which are missing data and how much (common missing data would probably be pretty bad for our model) and secondly, we need to figure out how these variables relate to eachother.  There is a date-time stamp in the data, and that could indicate we are dealing with a data series (or 4 data series, as I'm not sure what the MAC_CODE variable is.)  If our variables are a time series, this could substantially change the approach to this problem.","metadata":{}},{"cell_type":"code","source":"training_data.columns  #produce a list of the columns available as features in the dataset.","metadata":{"execution":{"iopub.status.busy":"2022-06-05T05:40:54.95803Z","iopub.execute_input":"2022-06-05T05:40:54.958636Z","iopub.status.idle":"2022-06-05T05:40:54.965648Z","shell.execute_reply.started":"2022-06-05T05:40:54.958591Z","shell.execute_reply":"2022-06-05T05:40:54.964747Z"},"trusted":true},"execution_count":null,"outputs":[]}]}