{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Flavours of Physics: Finding τ → μμμ (Kernels Only)  with EDA & ML (SVM+KNN+GNB+XGB)\n\n<img src=\"https://storage.googleapis.com/kaggle-competitions/kaggle/10014/logos/header.png?t=2018-06-20-19-58-34\" width=\"1000px\">\n\n\n### We got the better score\n* Accuracy XGBClassifier: 0.9999227202472952\n* Precision XGBClassifier: 0.9998466492869191\n* Recall XGBClassifier: 1.0\n* F1 Score XGBClassifier: 0.9999233187638984\n\n\n","metadata":{}},{"cell_type":"markdown","source":"\n### Data Description\nIn this competition, you are given a list of collision events and their properties. You will then predict whether a τ → 3μ decay happened in this collision. This τ → 3μ is currently assumed by scientists not to happen, and the goal of this competition is to discover τ → 3μ happening more frequently than scientists currently can understand.\n\n\nIt is challenging to design a machine learning problem for something you have never observed before. Scientists at CERN developed the following designs to achieve the goal.\n\n\n* training.csv\nThis is a labelled dataset (the label ‘signal’ being ‘1’ for signal events, ‘0’ for background events) to train the classifier. Signal events have been simulated, while background events are real data.\n\nThis real data is collected by the LHCb detectors observing collisions of accelerated particles with a specific mass range in which τ → 3μ can’t happen. We call these events “background” and label them 0.\n\n* test.csv\nThe test dataset has all the columns that training.csv has, except mass, production, min_ANNmuon, and signal. \n\nLink dataset\n\n[Here](https://www.kaggle.com/c/flavours-of-physics-kernels-only/data)\n","metadata":{}},{"cell_type":"code","source":"!pip install sweetviz","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:57:58.569467Z","iopub.execute_input":"2021-09-29T08:57:58.569864Z","iopub.status.idle":"2021-09-29T08:58:07.258094Z","shell.execute_reply.started":"2021-09-29T08:57:58.569826Z","shell.execute_reply":"2021-09-29T08:58:07.256495Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport numpy as np\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport seaborn as sns\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:46:31.975613Z","iopub.execute_input":"2021-09-29T08:46:31.976002Z","iopub.status.idle":"2021-09-29T08:46:32.985746Z","shell.execute_reply.started":"2021-09-29T08:46:31.975899Z","shell.execute_reply":"2021-09-29T08:46:32.984526Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train = pd.read_csv(\"../input/flavours-of-physics-kernels-only/training.csv.zip\")\ndf_test = pd.read_csv(\"../input/flavours-of-physics-kernels-only/test.csv.zip\")","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:46:33.602328Z","iopub.execute_input":"2021-09-29T08:46:33.602682Z","iopub.status.idle":"2021-09-29T08:46:58.428493Z","shell.execute_reply.started":"2021-09-29T08:46:33.602641Z","shell.execute_reply":"2021-09-29T08:46:58.427156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:46:58.430071Z","iopub.execute_input":"2021-09-29T08:46:58.430352Z","iopub.status.idle":"2021-09-29T08:46:58.493749Z","shell.execute_reply.started":"2021-09-29T08:46:58.430319Z","shell.execute_reply":"2021-09-29T08:46:58.492531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:12.337389Z","iopub.execute_input":"2021-09-28T17:48:12.338287Z","iopub.status.idle":"2021-09-28T17:48:12.572542Z","shell.execute_reply.started":"2021-09-28T17:48:12.338241Z","shell.execute_reply":"2021-09-28T17:48:12.571607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train= df_train[['IP', 'IPSig', 'isolatione', 'iso', 'ISO_SumBDT', 'p0_IsoBDT','p1_IsoBDT', 'SPDhits', 'signal']]","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:46:58.495416Z","iopub.execute_input":"2021-09-29T08:46:58.495798Z","iopub.status.idle":"2021-09-29T08:46:58.508160Z","shell.execute_reply.started":"2021-09-29T08:46:58.495748Z","shell.execute_reply":"2021-09-29T08:46:58.507369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2= df_test[['IP', 'IPSig', 'isolatione', 'iso', 'ISO_SumBDT', 'p0_IsoBDT','p1_IsoBDT', 'SPDhits', ]]","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:46:58.509967Z","iopub.execute_input":"2021-09-29T08:46:58.510455Z","iopub.status.idle":"2021-09-29T08:46:58.543131Z","shell.execute_reply.started":"2021-09-29T08:46:58.510408Z","shell.execute_reply":"2021-09-29T08:46:58.542462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:46:58.545431Z","iopub.execute_input":"2021-09-29T08:46:58.545790Z","iopub.status.idle":"2021-09-29T08:46:58.572394Z","shell.execute_reply.started":"2021-09-29T08:46:58.545753Z","shell.execute_reply":"2021-09-29T08:46:58.571373Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:47:04.136536Z","iopub.execute_input":"2021-09-29T08:47:04.137512Z","iopub.status.idle":"2021-09-29T08:47:04.163503Z","shell.execute_reply.started":"2021-09-29T08:47:04.137454Z","shell.execute_reply":"2021-09-29T08:47:04.162577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.info()","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:47:06.861615Z","iopub.execute_input":"2021-09-29T08:47:06.862527Z","iopub.status.idle":"2021-09-29T08:47:06.890265Z","shell.execute_reply.started":"2021-09-29T08:47:06.862476Z","shell.execute_reply":"2021-09-29T08:47:06.889005Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test.info()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:12.718254Z","iopub.execute_input":"2021-09-28T17:48:12.718547Z","iopub.status.idle":"2021-09-28T17:48:12.827735Z","shell.execute_reply.started":"2021-09-28T17:48:12.718508Z","shell.execute_reply":"2021-09-28T17:48:12.826674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train[\"signal\"].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:12.829111Z","iopub.execute_input":"2021-09-28T17:48:12.829438Z","iopub.status.idle":"2021-09-28T17:48:12.837977Z","shell.execute_reply.started":"2021-09-28T17:48:12.829399Z","shell.execute_reply":"2021-09-28T17:48:12.836972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:50:02.867280Z","iopub.execute_input":"2021-09-29T08:50:02.867664Z","iopub.status.idle":"2021-09-29T08:50:02.913840Z","shell.execute_reply.started":"2021-09-29T08:50:02.867622Z","shell.execute_reply":"2021-09-29T08:50:02.912532Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import sweetviz as sv\nTr_report1 = sv.analyze(df_train)\nTr_report1.show_notebook(w=\"90%\", h=\"full\")\nTr_report1.show_html('Tr_report1.html')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Tr_report1 = sv.analyze(df2)\nTr_report1.show_notebook(w=\"90%\", h=\"full\")\nTr_report1.show_html('Tr_report1.html')","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:51:06.322687Z","iopub.execute_input":"2021-09-29T08:51:06.323042Z","iopub.status.idle":"2021-09-29T08:51:18.709755Z","shell.execute_reply.started":"2021-09-29T08:51:06.323008Z","shell.execute_reply":"2021-09-29T08:51:18.708565Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df2","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:52:17.209290Z","iopub.execute_input":"2021-09-29T08:52:17.209649Z","iopub.status.idle":"2021-09-29T08:52:17.234694Z","shell.execute_reply.started":"2021-09-29T08:52:17.209615Z","shell.execute_reply":"2021-09-29T08:52:17.233556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['IP'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:54:52.147510Z","iopub.execute_input":"2021-09-29T08:54:52.147859Z","iopub.status.idle":"2021-09-29T08:54:52.828269Z","shell.execute_reply.started":"2021-09-29T08:54:52.147823Z","shell.execute_reply":"2021-09-29T08:54:52.826858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['IPSig'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:53:31.359523Z","iopub.execute_input":"2021-09-29T08:53:31.359919Z","iopub.status.idle":"2021-09-29T08:53:31.365576Z","shell.execute_reply.started":"2021-09-29T08:53:31.359879Z","shell.execute_reply":"2021-09-29T08:53:31.364290Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['isolatione'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['iso'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['ISO_SumBDT'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['p0_IsoBDT'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['p0_IsoBDT'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['p1_IsoBDT'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.distplot(df_train['SPDhits'],hist_kws={ \"linewidth\": 3,\"alpha\": 1, \"color\": \"r\"});","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.plot(figsize = (11, 25), subplots = True, linewidth = 0.8, color = \"r\")\nplt.xlabel('')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-29T08:55:39.432181Z","iopub.execute_input":"2021-09-29T08:55:39.433672Z","iopub.status.idle":"2021-09-29T08:55:42.609189Z","shell.execute_reply.started":"2021-09-29T08:55:39.433607Z","shell.execute_reply":"2021-09-29T08:55:42.607704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train['signal'].value_counts()\n\ndf_train['signal'].value_counts() * 100 / len(df_train)\n\n\nsns.countplot(x='signal', data=df_train, palette='viridis')","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:12.852732Z","iopub.execute_input":"2021-09-28T17:48:12.853083Z","iopub.status.idle":"2021-09-28T17:48:13.078913Z","shell.execute_reply.started":"2021-09-28T17:48:12.853039Z","shell.execute_reply":"2021-09-28T17:48:13.078169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.utils import resample\n\nnot_fraud = df_train[df_train['signal']== 0]\nfraud = df_train[df_train['signal'] == 1]\n\n#upsample minority\nfraud_upsampled = resample(fraud,\n                          replace=True, # sample with replacement\n                          n_samples=len(not_fraud), # match number in majority class\n                          random_state=27) # reproducible results\n\n#combine majority and upsampled minority\ndf_train = pd.concat([not_fraud, fraud_upsampled])\n\n# check new class counts\ndf_train['signal'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:13.081576Z","iopub.execute_input":"2021-09-28T17:48:13.081937Z","iopub.status.idle":"2021-09-28T17:48:13.182341Z","shell.execute_reply.started":"2021-09-28T17:48:13.081896Z","shell.execute_reply":"2021-09-28T17:48:13.181461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.countplot(x='signal', data=df_train, palette='viridis')","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:59:48.218619Z","iopub.execute_input":"2021-09-28T17:59:48.218983Z","iopub.status.idle":"2021-09-28T17:59:48.358994Z","shell.execute_reply.started":"2021-09-28T17:59:48.218947Z","shell.execute_reply":"2021-09-28T17:59:48.358001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:13.183598Z","iopub.execute_input":"2021-09-28T17:48:13.184275Z","iopub.status.idle":"2021-09-28T17:48:13.196311Z","shell.execute_reply.started":"2021-09-28T17:48:13.184234Z","shell.execute_reply":"2021-09-28T17:48:13.195324Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.describe()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:13.197706Z","iopub.execute_input":"2021-09-28T17:48:13.198576Z","iopub.status.idle":"2021-09-28T17:48:13.252492Z","shell.execute_reply.started":"2021-09-28T17:48:13.198524Z","shell.execute_reply":"2021-09-28T17:48:13.251525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.corr()['signal'].sort_values(ascending=False)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:13.254212Z","iopub.execute_input":"2021-09-28T17:48:13.254521Z","iopub.status.idle":"2021-09-28T17:48:13.278766Z","shell.execute_reply.started":"2021-09-28T17:48:13.254483Z","shell.execute_reply":"2021-09-28T17:48:13.277922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"z = df_train.drop('signal', axis=1)\nz.corrwith(df_train['signal']).plot(kind='bar', figsize=(15,10), color=['g'])\nplt.title(\"Correlation Matrix\")\nplt.xticks(size=15)\nplt.yticks(size=15)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:13.280227Z","iopub.execute_input":"2021-09-28T17:48:13.28072Z","iopub.status.idle":"2021-09-28T17:48:13.545772Z","shell.execute_reply.started":"2021-09-28T17:48:13.280668Z","shell.execute_reply":"2021-09-28T17:48:13.545149Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15,10))\nsns.heatmap(df_train.corr(), annot=True, cbar=False, fmt='.1f', cmap='summer')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:13.547225Z","iopub.execute_input":"2021-09-28T17:48:13.547728Z","iopub.status.idle":"2021-09-28T17:48:14.086519Z","shell.execute_reply.started":"2021-09-28T17:48:13.547676Z","shell.execute_reply":"2021-09-28T17:48:14.085751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15,10))\nsns.boxplot(data=df_train)\nplt.xticks(rotation=90)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:14.087722Z","iopub.execute_input":"2021-09-28T17:48:14.088112Z","iopub.status.idle":"2021-09-28T17:48:14.60386Z","shell.execute_reply.started":"2021-09-28T17:48:14.088068Z","shell.execute_reply":"2021-09-28T17:48:14.603222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:14.605068Z","iopub.execute_input":"2021-09-28T17:48:14.605326Z","iopub.status.idle":"2021-09-28T17:48:14.626029Z","shell.execute_reply.started":"2021-09-28T17:48:14.605296Z","shell.execute_reply":"2021-09-28T17:48:14.625277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.tight_layout()\nsns.boxplot(x=df_train[\"ISO_SumBDT\"], data=df_train)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:14.627565Z","iopub.execute_input":"2021-09-28T17:48:14.62782Z","iopub.status.idle":"2021-09-28T17:48:14.831961Z","shell.execute_reply.started":"2021-09-28T17:48:14.627791Z","shell.execute_reply":"2021-09-28T17:48:14.830689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.tight_layout()\nsns.boxplot(x=df_train[\"p0_IsoBDT\"], data=df_train)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:14.833378Z","iopub.execute_input":"2021-09-28T17:48:14.834092Z","iopub.status.idle":"2021-09-28T17:48:15.031816Z","shell.execute_reply.started":"2021-09-28T17:48:14.834054Z","shell.execute_reply":"2021-09-28T17:48:15.030868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.tight_layout()\nsns.boxplot(x=df_train[\"SPDhits\"], data=df_train)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.03296Z","iopub.execute_input":"2021-09-28T17:48:15.033887Z","iopub.status.idle":"2021-09-28T17:48:15.220743Z","shell.execute_reply.started":"2021-09-28T17:48:15.033849Z","shell.execute_reply":"2021-09-28T17:48:15.220013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.tight_layout()\nsns.boxplot(x=df_train[\"IPSig\"], data=df_train)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.222009Z","iopub.execute_input":"2021-09-28T17:48:15.222445Z","iopub.status.idle":"2021-09-28T17:48:15.408214Z","shell.execute_reply.started":"2021-09-28T17:48:15.222414Z","shell.execute_reply":"2021-09-28T17:48:15.407339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(df_train.skew())","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.41001Z","iopub.execute_input":"2021-09-28T17:48:15.410597Z","iopub.status.idle":"2021-09-28T17:48:15.424346Z","shell.execute_reply.started":"2021-09-28T17:48:15.41055Z","shell.execute_reply":"2021-09-28T17:48:15.423612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\nx1 = df_train.drop('signal', axis=1)\ny = df_train['signal']","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.425433Z","iopub.execute_input":"2021-09-28T17:48:15.425989Z","iopub.status.idle":"2021-09-28T17:48:15.488538Z","shell.execute_reply.started":"2021-09-28T17:48:15.425958Z","shell.execute_reply":"2021-09-28T17:48:15.487509Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x1","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.489704Z","iopub.execute_input":"2021-09-28T17:48:15.490459Z","iopub.status.idle":"2021-09-28T17:48:15.516883Z","shell.execute_reply.started":"2021-09-28T17:48:15.490423Z","shell.execute_reply":"2021-09-28T17:48:15.515907Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.518332Z","iopub.execute_input":"2021-09-28T17:48:15.51895Z","iopub.status.idle":"2021-09-28T17:48:15.525547Z","shell.execute_reply.started":"2021-09-28T17:48:15.518922Z","shell.execute_reply":"2021-09-28T17:48:15.524652Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from imblearn.over_sampling import SMOTE\n\nx1 = df_train.drop('signal', axis=1)\ny = df_train['signal']\n\n# setting up testing and training sets\nx_train, x_test, y_train, y_test = train_test_split(x1, y, test_size=0.25, random_state=27)\n\nsm = SMOTE(random_state=27)\n\nx1, y = sm.fit_resample(x1, y)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.526864Z","iopub.execute_input":"2021-09-28T17:48:15.527566Z","iopub.status.idle":"2021-09-28T17:48:15.697751Z","shell.execute_reply.started":"2021-09-28T17:48:15.527525Z","shell.execute_reply":"2021-09-28T17:48:15.697099Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import accuracy_score, confusion_matrix, f1_score, classification_report, roc_curve, roc_auc_score, auc, precision_recall_curve, precision_score, recall_score\n","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.698772Z","iopub.execute_input":"2021-09-28T17:48:15.699466Z","iopub.status.idle":"2021-09-28T17:48:15.704231Z","shell.execute_reply.started":"2021-09-28T17:48:15.699431Z","shell.execute_reply":"2021-09-28T17:48:15.703151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.svm import SVC\nsvm= SVC(random_state=2)\nsvm.fit(x_train, y_train)\ny_pred_svm = svm.predict(x_test)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:48:15.705402Z","iopub.execute_input":"2021-09-28T17:48:15.705645Z","iopub.status.idle":"2021-09-28T17:49:21.191117Z","shell.execute_reply.started":"2021-09-28T17:48:15.705617Z","shell.execute_reply":"2021-09-28T17:49:21.190243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_svm = svm.predict(x_test)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:21.196183Z","iopub.execute_input":"2021-09-28T17:49:21.198662Z","iopub.status.idle":"2021-09-28T17:49:35.181838Z","shell.execute_reply.started":"2021-09-28T17:49:21.198613Z","shell.execute_reply":"2021-09-28T17:49:35.179806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Accuracy SVC:\", accuracy_score(y_test, y_pred_svm))\nprint(\"Precision SVC:\", precision_score(y_test, y_pred_svm))\nprint(\"Recall SVC:\", recall_score(y_test, y_pred_svm))\nprint(\"F1 Score SVC:\", f1_score(y_test, y_pred_svm))","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:35.183773Z","iopub.execute_input":"2021-09-28T17:49:35.184177Z","iopub.status.idle":"2021-09-28T17:49:35.257828Z","shell.execute_reply.started":"2021-09-28T17:49:35.184147Z","shell.execute_reply":"2021-09-28T17:49:35.255882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.neighbors import KNeighborsClassifier\nKNN = KNeighborsClassifier(algorithm='auto', leaf_size=300, metric='minkowski',\n                     metric_params=None, n_jobs=None, n_neighbors=100, p=2,\n                     weights='uniform')\nKNN.fit(x1, y)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:35.259817Z","iopub.execute_input":"2021-09-28T17:49:35.260101Z","iopub.status.idle":"2021-09-28T17:49:35.366071Z","shell.execute_reply.started":"2021-09-28T17:49:35.260068Z","shell.execute_reply":"2021-09-28T17:49:35.364708Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_KNN = KNN.predict(x_test)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:35.37066Z","iopub.execute_input":"2021-09-28T17:49:35.37091Z","iopub.status.idle":"2021-09-28T17:49:37.115666Z","shell.execute_reply.started":"2021-09-28T17:49:35.370885Z","shell.execute_reply":"2021-09-28T17:49:37.111281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Accuracy KNeighborsClassifier:\", accuracy_score(y_test, y_pred_KNN))\nprint(\"Precision KNeighborsClassifier:\", precision_score(y_test, y_pred_KNN))\nprint(\"Recall KNeighborsClassifier:\", recall_score(y_test, y_pred_KNN))\nprint(\"F1 Score KNeighborsClassifier:\", f1_score(y_test, y_pred_KNN))","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:37.12255Z","iopub.execute_input":"2021-09-28T17:49:37.124664Z","iopub.status.idle":"2021-09-28T17:49:37.172018Z","shell.execute_reply.started":"2021-09-28T17:49:37.124602Z","shell.execute_reply":"2021-09-28T17:49:37.1709Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.naive_bayes import GaussianNB\nGNB = GaussianNB(priors=None, var_smoothing=1e-08)\nGNB.fit(x_train, y_train)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:37.179072Z","iopub.execute_input":"2021-09-28T17:49:37.182873Z","iopub.status.idle":"2021-09-28T17:49:37.214492Z","shell.execute_reply.started":"2021-09-28T17:49:37.182824Z","shell.execute_reply":"2021-09-28T17:49:37.213423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_GNB = GNB.predict(x_test)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:37.224505Z","iopub.execute_input":"2021-09-28T17:49:37.225061Z","iopub.status.idle":"2021-09-28T17:49:37.23721Z","shell.execute_reply.started":"2021-09-28T17:49:37.225017Z","shell.execute_reply":"2021-09-28T17:49:37.235266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Accuracy GaussianNB:\", accuracy_score(y_test, y_pred_GNB))\nprint(\"Precision GaussianNB:\", precision_score(y_test, y_pred_GNB))\nprint(\"Recall GaussianNB:\", recall_score(y_test, y_pred_GNB))\nprint(\"F1 Score GaussianNB:\", f1_score(y_test, y_pred_GNB))","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:37.240091Z","iopub.execute_input":"2021-09-28T17:49:37.240579Z","iopub.status.idle":"2021-09-28T17:49:37.287902Z","shell.execute_reply.started":"2021-09-28T17:49:37.240541Z","shell.execute_reply":"2021-09-28T17:49:37.28692Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from xgboost import XGBClassifier\nxgb = XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n              colsample_bynode=1, colsample_bytree=1, gamma=0, gpu_id=-1,\n              importance_type='gain', interaction_constraints='',\n              learning_rate=0.300000012, max_delta_step=0, max_depth=40,\n              min_child_weight=1, monotone_constraints='()',\n              n_estimators=1000, n_jobs=100, num_parallel_tree=1, random_state=0,\n              reg_alpha=2, reg_lambda=1, scale_pos_weight=1, subsample=1,\n              tree_method='exact', validate_parameters=1, verbosity=None)\nxgb.fit(x1, y)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:49:37.289482Z","iopub.execute_input":"2021-09-28T17:49:37.290087Z","iopub.status.idle":"2021-09-28T17:53:53.095568Z","shell.execute_reply.started":"2021-09-28T17:49:37.290043Z","shell.execute_reply":"2021-09-28T17:53:53.094694Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_xgb = xgb.predict(x_test)\n","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:54:35.033405Z","iopub.execute_input":"2021-09-28T17:54:35.034134Z","iopub.status.idle":"2021-09-28T17:54:35.323645Z","shell.execute_reply.started":"2021-09-28T17:54:35.03409Z","shell.execute_reply":"2021-09-28T17:54:35.322822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Accuracy XGBClassifier:\", accuracy_score(y_test, y_pred_xgb))\nprint(\"Precision XGBClassifier:\", precision_score(y_test, y_pred_xgb))\nprint(\"Recall XGBClassifier:\", recall_score(y_test, y_pred_xgb))\nprint(\"F1 Score XGBClassifier:\", f1_score(y_test, y_pred_xgb))","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:54:39.540753Z","iopub.execute_input":"2021-09-28T17:54:39.541288Z","iopub.status.idle":"2021-09-28T17:54:39.573931Z","shell.execute_reply.started":"2021-09-28T17:54:39.541246Z","shell.execute_reply":"2021-09-28T17:54:39.573032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_df_tset = xgb.predict(x1)\ny_pred_df_tset","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:55:36.494064Z","iopub.execute_input":"2021-09-28T17:55:36.494391Z","iopub.status.idle":"2021-09-28T17:55:37.483616Z","shell.execute_reply.started":"2021-09-28T17:55:36.494361Z","shell.execute_reply":"2021-09-28T17:55:37.482723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pd.read_csv('../input/flavours-of-physics-kernels-only/sample_submission.csv.zip')\nsub[:51758]","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:55:39.773108Z","iopub.execute_input":"2021-09-28T17:55:39.773717Z","iopub.status.idle":"2021-09-28T17:55:40.220823Z","shell.execute_reply.started":"2021-09-28T17:55:39.773657Z","shell.execute_reply":"2021-09-28T17:55:40.219377Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub[:51758]['prediction'] = y_pred_df_tset\nsub[:51758].to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2021-09-28T17:55:43.154847Z","iopub.execute_input":"2021-09-28T17:55:43.155578Z","iopub.status.idle":"2021-09-28T17:55:43.359668Z","shell.execute_reply.started":"2021-09-28T17:55:43.155537Z","shell.execute_reply":"2021-09-28T17:55:43.358853Z"},"trusted":true},"execution_count":null,"outputs":[]}]}