{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Kaggle: Empowering Data Enthusiasts to Learn by Doing and Access the Best Datasets","metadata":{}},{"cell_type":"markdown","source":"Kaggle is an online platform to learn, wander(explore), teach, play (with data),compete, build models (ML models), and collaborate with other data enthusiasts. Kaggle has become a recognized platform for showcasing data science skills and attracting the attention of potential employers. Participating in Kaggle competitions and sharing notable contributions can serve as a valuable addition to one's data science portfolio and enhance career prospects.\n","metadata":{}},{"cell_type":"markdown","source":" Life of a data enthusiasts be like
\n\n1. Search mode\n2. Hack mode - Learn by doing\n3. Compete mode\n4. Fun mode\n5. Loop back to Search mode\n","metadata":{}},{"cell_type":"markdown","source":"Here’s how kaggle helps data enthusiasts to complete their life cycle\n\n\n1. Datasets - Search Mode
\n\nData enthusiasts life is incomplete without data. Also it is difficult to get real world data for free. Kaggle comes here to help all individuals by hosting a vast collection of datasets from various domains, including finance, healthcare, sports, and more. These datasets are contributed by the Kaggle community and are available for free download, allowing users to explore and analyze real-world data.\n\nImage credits: Google\n\n","metadata":{}},{"cell_type":"markdown","source":"2. Learn - Hack mode
\n\nKaggle offers learning resources to help individuals enhance their data science skills. This includes tutorials, courses, and notebooks that cover a wide range of topics and techniques. Users can learn from experts, explore coding examples, and collaborate with others to deepen their understanding of data science concepts.\n\nImage credits: Google\n\n","metadata":{}},{"cell_type":"markdown","source":"3. Play - Compete mode
\n\nKaggle is renowned for its data science competitions. Organizations and individuals can host competitions on Kaggle, providing participants with datasets and problem statements to solve. Competitions cover diverse topics, ranging from image classification and natural language processing to predictive modeling and recommendation systems.\n\nImage credits: Google\n\n","metadata":{}},{"cell_type":"markdown","source":" 4. Publish and cherish - Fun mode
\n\nKaggle kernels are popular feature that allows user to create and publish their work with the data community. It is an interactive notebook that provides option for anyone to plug and play instantly.\n\nKaggle has a vibrant data scientist and machine learning community. Users can participate in any discussion, ask questions, share their work and get review for it. This provides opportunity to collaborate with other data enthusiasts.\n\nImage credits: Google\n\n\n","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-06-26T09:33:15.924831Z","iopub.execute_input":"2023-06-26T09:33:15.926393Z","iopub.status.idle":"2023-06-26T09:33:15.939469Z","shell.execute_reply.started":"2023-06-26T09:33:15.926309Z","shell.execute_reply":"2023-06-26T09:33:15.937983Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"kg_wr = pd.read_csv('/kaggle/input/2023-kaggle-ai-report/kaggle_writeups_20230510.csv')","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:33:22.326548Z","iopub.execute_input":"2023-06-26T09:33:22.327605Z","iopub.status.idle":"2023-06-26T09:33:22.552812Z","shell.execute_reply.started":"2023-06-26T09:33:22.327562Z","shell.execute_reply":"2023-06-26T09:33:22.551494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"kg_wr.info()","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:33:24.786479Z","iopub.execute_input":"2023-06-26T09:33:24.786876Z","iopub.status.idle":"2023-06-26T09:33:24.808709Z","shell.execute_reply.started":"2023-06-26T09:33:24.786847Z","shell.execute_reply":"2023-06-26T09:33:24.807391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# drop null fields\nkg_wr.dropna(inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:26:14.840001Z","iopub.execute_input":"2023-06-26T09:26:14.840514Z","iopub.status.idle":"2023-06-26T09:26:14.857822Z","shell.execute_reply.started":"2023-06-26T09:26:14.840470Z","shell.execute_reply":"2023-06-26T09:26:14.856760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"kg_wr.head()","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:26:14.861260Z","iopub.execute_input":"2023-06-26T09:26:14.862271Z","iopub.status.idle":"2023-06-26T09:26:14.881720Z","shell.execute_reply.started":"2023-06-26T09:26:14.862231Z","shell.execute_reply":"2023-06-26T09:26:14.880345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" Analysis of competition which had most write ups
","metadata":{}},{"cell_type":"code","source":"sns.countplot(y='Title of Competition',data=kg_wr, order=kg_wr['Title of Competition'].value_counts().iloc[:5].index)","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:26:14.883393Z","iopub.execute_input":"2023-06-26T09:26:14.884085Z","iopub.status.idle":"2023-06-26T09:26:15.137253Z","shell.execute_reply.started":"2023-06-26T09:26:14.884050Z","shell.execute_reply":"2023-06-26T09:26:15.135716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Availability of dataset","metadata":{}},{"cell_type":"code","source":"ds = pd.read_csv('/kaggle/input/kaggle-dataset/kaggle-preprocessed.csv')","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:36:44.496351Z","iopub.execute_input":"2023-06-26T09:36:44.496855Z","iopub.status.idle":"2023-06-26T09:36:44.594968Z","shell.execute_reply.started":"2023-06-26T09:36:44.496818Z","shell.execute_reply":"2023-06-26T09:36:44.593817Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds.info()","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:36:50.597728Z","iopub.execute_input":"2023-06-26T09:36:50.598119Z","iopub.status.idle":"2023-06-26T09:36:50.646476Z","shell.execute_reply.started":"2023-06-26T09:36:50.598091Z","shell.execute_reply":"2023-06-26T09:36:50.643515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds.head()","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:52:42.655442Z","iopub.execute_input":"2023-06-26T09:52:42.656244Z","iopub.status.idle":"2023-06-26T09:52:42.677481Z","shell.execute_reply.started":"2023-06-26T09:52:42.656209Z","shell.execute_reply":"2023-06-26T09:52:42.676556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Top most used datasets at kaggle:\", len(ds))","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:52:46.395176Z","iopub.execute_input":"2023-06-26T09:52:46.395635Z","iopub.status.idle":"2023-06-26T09:52:46.401183Z","shell.execute_reply.started":"2023-06-26T09:52:46.395601Z","shell.execute_reply":"2023-06-26T09:52:46.400252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds['Day'].value_counts(normalize=True).plot(kind='bar')\nplt.title('Day which had maximum dataset upload')","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:45:15.052051Z","iopub.execute_input":"2023-06-26T09:45:15.052554Z","iopub.status.idle":"2023-06-26T09:45:15.461836Z","shell.execute_reply.started":"2023-06-26T09:45:15.052517Z","shell.execute_reply":"2023-06-26T09:45:15.460445Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.to_datetime(ds['Date']).dt.year.value_counts(normalize=True).plot(kind='bar')\nplt.title('Year which had maximum dataset uploads')","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:45:48.300628Z","iopub.execute_input":"2023-06-26T09:45:48.301062Z","iopub.status.idle":"2023-06-26T09:45:48.665125Z","shell.execute_reply.started":"2023-06-26T09:45:48.301030Z","shell.execute_reply":"2023-06-26T09:45:48.663577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds[ds['Upvotes']==ds['Upvotes'].max()]","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:57:04.733666Z","iopub.execute_input":"2023-06-26T09:57:04.734090Z","iopub.status.idle":"2023-06-26T09:57:04.755875Z","shell.execute_reply.started":"2023-06-26T09:57:04.734059Z","shell.execute_reply":"2023-06-26T09:57:04.754633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Sentiment analysis of write ups","metadata":{}},{"cell_type":"code","source":"from nltk.sentiment import SentimentIntensityAnalyzer","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:26:15.613493Z","iopub.execute_input":"2023-06-26T09:26:15.613820Z","iopub.status.idle":"2023-06-26T09:26:15.623445Z","shell.execute_reply.started":"2023-06-26T09:26:15.613792Z","shell.execute_reply":"2023-06-26T09:26:15.622136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sia = SentimentIntensityAnalyzer()","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:26:15.625553Z","iopub.execute_input":"2023-06-26T09:26:15.626038Z","iopub.status.idle":"2023-06-26T09:26:15.647723Z","shell.execute_reply.started":"2023-06-26T09:26:15.626006Z","shell.execute_reply":"2023-06-26T09:26:15.646294Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tone = {\n \"neu\": 0,\n \"pos\": 0,\n \"neg\": 0\n}\ndef predict_tone_of_wrtup(df):\n res = sia.polarity_scores(df['Writeup'])\n tone[\"neu\"] += res[\"neu\"]\n tone[\"pos\"] += res[\"pos\"]\n tone[\"neg\"] += res[\"neg\"]","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:26:15.649403Z","iopub.execute_input":"2023-06-26T09:26:15.649827Z","iopub.status.idle":"2023-06-26T09:26:15.662424Z","shell.execute_reply.started":"2023-06-26T09:26:15.649795Z","shell.execute_reply":"2023-06-26T09:26:15.660873Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"kg_wr.apply(predict_tone_of_wrtup, axis=1)","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:26:15.664156Z","iopub.execute_input":"2023-06-26T09:26:15.664656Z","iopub.status.idle":"2023-06-26T09:26:44.987541Z","shell.execute_reply.started":"2023-06-26T09:26:15.664619Z","shell.execute_reply":"2023-06-26T09:26:44.986368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x = list(tone.keys())\ny = [float(tone[k]) for k in x]\nplt.bar(x=x,height=y)\nplt.title(\"Nature of writeup\")","metadata":{"execution":{"iopub.status.busy":"2023-06-26T09:26:44.991643Z","iopub.execute_input":"2023-06-26T09:26:44.992838Z","iopub.status.idle":"2023-06-26T09:26:45.218628Z","shell.execute_reply.started":"2023-06-26T09:26:44.992799Z","shell.execute_reply":"2023-06-26T09:26:45.217480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df = pd.read_csv(\"/kaggle/input/2023-kaggle-ai-report/sample_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-06-26T10:04:43.931353Z","iopub.execute_input":"2023-06-26T10:04:43.932180Z","iopub.status.idle":"2023-06-26T10:04:43.948533Z","shell.execute_reply.started":"2023-06-26T10:04:43.932145Z","shell.execute_reply":"2023-06-26T10:04:43.947430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-06-26T10:05:41.236821Z","iopub.execute_input":"2023-06-26T10:05:41.237281Z","iopub.status.idle":"2023-06-26T10:05:41.248060Z","shell.execute_reply.started":"2023-06-26T10:05:41.237247Z","shell.execute_reply":"2023-06-26T10:05:41.247072Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.loc[0]['value']='Kaggle Competition'\nsubmission_df.loc[1]['value']='https://www.kaggle.com/code/lrsowmya/kaggle-empowering-data-enthusiasts-to-learn-by-do'\nsubmission_df.loc[2]['value']='https://www.kaggle.com/code/poojach7611/tabular-and-time-series-data/comments#2318350'\nsubmission_df.loc[3]['value']='https://www.kaggle.com/code/jocelyndumlao/ai-report-kaggle-competitions/comments#2318359'\nsubmission_df.loc[4]['value']='https://www.kaggle.com/code/mpwolke/leave-a-team-a-rule-to-be-broken/comments#2318366'\nsubmission_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2023-06-26T10:21:49.133708Z","iopub.execute_input":"2023-06-26T10:21:49.134141Z","iopub.status.idle":"2023-06-26T10:21:49.151176Z","shell.execute_reply.started":"2023-06-26T10:21:49.134111Z","shell.execute_reply":"2023-06-26T10:21:49.149318Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It is evident that most of the writeups have neutral tone. **This suggests Kaggle as Harassement free and friendly environment for data enthusiasts**","metadata":{}},{"cell_type":"markdown","source":"In this notebook we have analysed three different features of kaggle\n\n1.Competitions\n\n2.Dataset\n\n3.Kaggle community\n\nThese features make Kaggle as a platform which empowers data enthusiasts to learn by playing.","metadata":{}}]}