{"cells":[{"metadata":{},"cell_type":"markdown","source":"> ## Term Project Challenge!\n\nIn this tutorial, we shall train a model for detection of fake news. We shall be using Naive Bayes algorithm as a starting point. If you have seen my lecture videos, you now understand how Naive bayes works. First of all we shall import all required libraries.","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"nbpresent":{"id":"45118ea8-187a-4da2-b6ce-63e6b65830fb"}},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nfrom sklearn.feature_extraction.text import CountVectorizer\nfrom sklearn.feature_extraction.text import TfidfTransformer\nfrom sklearn.pipeline import Pipeline\n\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.naive_bayes import MultinomialNB","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We shall now read the training and test dataset and define Id as an index column.","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"nbpresent":{"id":"015cd8ce-190e-43fb-8efa-7f4ab159206e"}},"cell_type":"code","source":"dataset = pd.read_csv('../input/classifying-the-fake-news/training.csv',index_col='Id')\ntest = pd.read_csv('../input/classifying-the-fake-news/test.csv',index_col='Id')\ndataset.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Okay, let use text field for training our model and label is the feature that we have to predict. That is zero means not fake and 1 means fake news.","execution_count":null},{"metadata":{"nbpresent":{"id":"10b545b4-997e-4133-bb9f-e9ca30908860"},"trusted":true},"cell_type":"code","source":"X_train = dataset.loc[:, \"text\"]\ny_train = dataset.loc[:, \"label\"]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's create a simpel pipeline. First vectorize the text in every news article, then, we shall use tfidf weighting and, finally, we shall use naive bayes in our pipeline.","execution_count":null},{"metadata":{"nbpresent":{"id":"290139d8-853b-4926-8fc5-6696fa356fc6"},"trusted":true},"cell_type":"code","source":"pipe = Pipeline([('vect', CountVectorizer()),\n                 ('tfidf', TfidfTransformer()),\n                 ('model', MultinomialNB())])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So, our pipeline is ready now. Let's now train the model. It will take some time so be patient.","execution_count":null},{"metadata":{"nbpresent":{"id":"68a893f1-aeec-43db-ab9d-66b5625e6d35"},"trusted":true},"cell_type":"code","source":"model = pipe.fit(X_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Great! we are done with training. Let us now test the data. In the test data, we don't have labels. You will be submitting the labels against each Id as a submission file on this portal and it will show you the scores.","execution_count":null},{"metadata":{"nbpresent":{"id":"e214bd82-9de8-4dfd-ab60-02e934bfb0bb"},"trusted":true},"cell_type":"code","source":"X_test = test['text']\npredictions = model.predict(X_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"It's time to create a dataframe appropriate for submission.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.DataFrame({\n        \"Id\": test.index,\n        \"Predicted\": predictions\n    })","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"Finally, save the submission file. Download this file from the right side in **kaggle/working** directory and then upload and make submission.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.to_csv('submission.csv',index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}