{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Categorical Feature Encoding Challenge 2 WITH MEMES&CATS\n### Come on, come learn too!\n![](https://media1.giphy.com/media/E0cyxhawhe9dm/200.webp?cid=790b7611f3ad87c38ca93cd44145076323983ac377bddc6c&rid=200.webp)\n[Nyan Cat [original]](https://youtu.be/QH2-TGUlwu4)\n\n[Crislânio Macêdo](https://medium.com/sapere-aude-tech) - **Kernel Created** - January, 08th, 2019. **Last Update**- January, 08th, 2019\n\n Learn Logistic Regression with Cats 🐱: [ 😵 Logistic Regression & 😺 Random Cats ](https://www.kaggle.com/caesarlupum/learn-logistic-regression-with-random-cats)\n\n\n\n- [**Github**](https://github.com/crislanio)\n- [**Linkedin**](https://www.linkedin.com/in/crislanio/)\n- [**Medium**](https://medium.com/sapere-aude-tech)\n- [**Quora**](https://www.quora.com/profile/Crislanio)\n- [**Ensina.AI**](https://medium.com/ensina-ai/an%C3%A1lise-dos-dados-abertos-do-governo-federal-ba65af8c421c)\n- [**Hackerrank**](https://www.hackerrank.com/crislanio_ufc?hr_r=1)\n- [**Blog**](https://medium.com/@crislanio.ufc)\n- [**Personal Page**](https://crislanio.wordpress.com/about)\n- [**Twitter**](https://twitter.com/crs_macedo)\n\n----------\n\n"},{"metadata":{},"cell_type":"markdown","source":"\n# Categorical Feature Encoding Challenge II\n## Binary classification, with every feature a categorical (and interactions!)\n## About this Competition\n\n![](https://media.giphy.com/media/H4DjXQXamtTiIuCcRU/giphy.gif)\n\n> #### In this competition, you will be predicting the probability [0, 1] of a binary target column.\n\nThe data contains binary features (bin_*), nominal features (nom_*), ordinal features (ord_*) as well as (potentially cyclical) day (of the week) and month features. The string ordinal features ord_{3-5} are lexically ordered according to string.ascii_letters.\nSince the purpose of this competition is to explore various encoding strategies, the data has been simplified in that (1) there are no missing values, and (2) the test set does not contain any unseen feature values (See this). (Of course, in real-world settings both of these factors are often important to consider!)\n\n#### Files\n- train.csv - the training set\n- test.csv - the test set; you must make predictions against this data\n- sample_submission.csv - a sample submission file in the correct format\n\n"},{"metadata":{},"cell_type":"markdown","source":"###  **I'm ready** !\n\n![](https://media0.giphy.com/media/VbnUQpnihPSIgIXuZv/giphy.webp?cid=790b7611f3ad87c38ca93cd44145076323983ac377bddc6c&rid=giphy.webp)\n\n[Dank Cat Memes Compilation](https://youtu.be/jNE5A86OryY)\n"},{"metadata":{},"cell_type":"markdown","source":"# Imports\n> We are using a typical data science stack: ``numpy``, ``pandas``, ``sklearn``, ``matplotlib``"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport gc\nimport warnings\nwarnings.filterwarnings(\"ignore\", category=DeprecationWarning)\nwarnings.filterwarnings(\"ignore\", category=UserWarning)\nwarnings.filterwarnings(\"ignore\", category=FutureWarning)\nfrom IPython.display import HTML\npd.set_option('max_columns', 50)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### So Excited ! I can't Wait !\n![](https://media0.giphy.com/media/JIX9t2j0ZTN9S/giphy.webp?cid=790b7611f3ad87c38ca93cd44145076323983ac377bddc6c&rid=giphy.webp)\n\n[MEOWWWW | Yelling Cat Meme](https://youtu.be/hDUJIPYEcLU?t=41)"},{"metadata":{},"cell_type":"markdown","source":"## Logistic Regression (Binary Classification)"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"HTML('<iframe width=\"921\" height=\"587\" src=\"https://www.youtube.com/embed/zM4VZR0px8E\" frameborder=\"0\" allow=\"accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>' )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Read in Data\n> First, we can list all the available data files. There are a total of 3 files: 1 main file for training (with target) 1 main file for testing (without the target), 1 example submission file."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train = pd.read_csv('../input/cat-in-the-dat-ii/train.csv')\ntest = pd.read_csv('../input/cat-in-the-dat-ii/test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"labels = train.pop('target')\ntrain_id = train.pop(\"id\")\ntest_id = test.pop(\"id\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Catching Cats\n![Catching Cats](https://miro.medium.com/proxy/0*5KosvsbnMQSOFABh.gif)\n[DANKEST CAT MEMES](https://youtu.be/AehCZYWweFo)"},{"metadata":{},"cell_type":"markdown","source":"# Get Dummies\n> Convert categorical variable into dummy/indicator variables [1]"},{"metadata":{"trusted":true},"cell_type":"code","source":"labels = labels.values\ndata = pd.concat([train, test])\n\ncolumns = [i for i in data.columns]\ndummies = pd.get_dummies(data,columns=columns, drop_first=True,sparse=True)\n\ntrain = dummies.iloc[:train.shape[0], :]\ntest = dummies.iloc[train.shape[0]:, :]\n\ndel dummies,data\ngc.collect()\n\ntrain = train.sparse.to_coo().tocsr()\ntest = test.sparse.to_coo().tocsr()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"![](https://media0.giphy.com/media/Nm8ZPAGOwZUQM/giphy.gif?cid=790b761175d6f9511d6eff1d6b0eeaaeb300715132180238&rid=giphy.gif)\n[BEWARE! Extremely angry cats!](https://youtu.be/HBFBmvt6ZIU?t=42)"},{"metadata":{},"cell_type":"markdown","source":"\n# Make Prediction\n## Logistic Regression"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.linear_model import LogisticRegression\nlr = LogisticRegression(C=0.20, solver=\"lbfgs\", tol=0.020, max_iter=2020)\n\nlr.fit(train, labels)\nlr_pred = lr.predict_proba(train)[:, 1]\nlr_pred","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Its easy, maybe not !\n![](https://media1.giphy.com/media/l0MYNB04rBb51QNtC/giphy.webp?cid=790b761175d6f9511d6eff1d6b0eeaaeb300715132180238&rid=giphy.webp)\n[Cat attacking people Compilation](https://youtu.be/HrgbZlE7q7k?t=2)\n"},{"metadata":{},"cell_type":"markdown","source":"# Evaluate"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.metrics import roc_auc_score\nscore = roc_auc_score(labels, lr_pred)\n\nprint(f\"{score:.6f}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### I'm win\n\n![](https://media0.giphy.com/media/1iu8uG2cjYFZS6wTxv/giphy.webp?cid=790b761175d6f9511d6eff1d6b0eeaaeb300715132180238&rid=giphy.webp)\n[DANKEST CAT MEMES#2](https://youtu.be/HEX2MK9bcos)\n"},{"metadata":{},"cell_type":"markdown","source":"## Sample Submission"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"submission = pd.read_csv(\"/kaggle/input/cat-in-the-dat-ii/sample_submission.csv\")\nsubmission[\"id\"] = test_id\nsubmission[\"target\"] = lr.predict_proba(test)[:, 1]\nsubmission.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"submission.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# General findinds\n\n#### Read Material from Competition  \n- [Best material from all comps!](https://www.kaggle.com/c/cat-in-the-dat-ii/discussion/124773) by [Brunhs](https://www.kaggle.com/brunhs)\n\n- [CATEGORICAL MATERIAL SURVEY🐱 & Record Linkage](https://www.kaggle.com/c/cat-in-the-dat-ii/discussion/124773) by [Caesar Lupum](https://www.kaggle.com/caesarlupum)\n\n#### Categorical feature encoding  \n\n- [Categorical feature encoding with TensorFlow](https://www.kaggle.com/c/cat-in-the-dat-ii/discussion/124217) by [Luca Massaron](https://www.kaggle.com/lucamassaron)\n\n\n"},{"metadata":{},"cell_type":"markdown","source":"# References\n- [Pandas Get Dummies](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.get_dummies.html)\n- [Kernel Logistic Regression by Martin Matta](https://www.kaggle.com/martin1234567890/logistic-regression)\n"},{"metadata":{},"cell_type":"markdown","source":"<html>\n<body>\n<p><font size=\"4\" color=\"blue\">If you like my kernel please consider upvoting it</font></p>\n<p><font size=\"3\" color=\"purple\">Remember the upvote button is next to the fork button, and it's free too! ;)</font></p>\n</body>\n</html>"},{"metadata":{},"cell_type":"markdown","source":"# Final"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}