{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Kaggle AI Report - Top YouTubers in Data Science Field\n\nI am sure most of us, at some point in our data science journey, have stumbled upon YouTube videos in our quest for knowledge. YouTube has become an invaluable resource for individuals interested in the field of data science, providing a platform for experts and enthusiasts to share their expertise, insights, and tutorials. \n\nWhen it comes to the top YouTubers in the data science field, several names stand out for their exceptional content and contributions. These individuals have amassed substantial followings and have established themselves as authorities in the data science community. They offer a diverse range of perspectives, expertise, and teaching styles, ensuring that there is something for everyone.\n\nSo, let's dive in and discover the captivating world of data science YouTubers!","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"code","source":"import pandas as pd\npd.set_option(\"display.max_rows\", 5)\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom IPython.display import JSON, display, HTML\nfrom googleapiclient.discovery import build\nfrom kaggle_secrets import UserSecretsClient\nimport datetime\nimport re\nimport ast\nimport emoji\nimport warnings\nwarnings.filterwarnings('ignore')\nimport calendar\nimport nltk\nfrom nltk.tokenize import word_tokenize\nfrom nltk.probability import FreqDist\nfrom nltk.corpus import stopwords\n# Download NLTK data\nnltk.download('punkt')\nnltk.download('stopwords')\nfrom PIL import Image\nfrom wordcloud import WordCloud, STOPWORDS, ImageColorGenerator\nfrom transformers import AutoModelForSequenceClassification, AutoTokenizer, AutoConfig, pipeline\nfrom scipy.special import softmax\nfrom collections import Counter\n\ndef scrolling(df):\n    display(HTML(\"<div style='height: 200px; overflow: auto; width: fit-content'>\" +\n                 df.to_html() + #df.style.render() +\n                 \"</div>\"))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-15T06:00:19.988485Z","iopub.execute_input":"2023-07-15T06:00:19.988985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Setting up YouTube api key and list of YouTubers\n\nHere i am using the YouTube API to scrape data from Youtube channels subscribed by me personally. Had to do some research and added a dozen more. (may have to omit some of the youtube channels with more than 1000 videos 😂)\n\n(Expand to see the codes!)","metadata":{}},{"cell_type":"code","source":"secret_label = \"youtube-api\"\napi_key = UserSecretsClient().get_secret(secret_label)\n\nchannel_ids = ['UCJQJAI7IjbLcpsjWdSzYz0Q', # @Thuvu5\n               'UCxladMszXan-jfgzyeIMyvw', # @robmulla\n               'UCiT9RITQ9PW6BhXK0y2jaeg', # @KenJee_ds\n               'UCNU_lfiiWBdtULKOw6X0Dig', # @krishnaik06\n               'UCKMjvg6fB6WS5WrPtbV4F5g', # @KylieYYing\n               'UCWN3xxRkmTPmbKwht9FuE5A', # @SirajRaval\n               'UCR68xPzQK7Etq2YfJ6Oe_2A', # @thedatajanitor9537\n               'UCtslD4DGH6PKyG_1gFAX7sg', # @AAmini\n               'UCcIXc5mJsHVYTZR1maL5l9w', # @Deeplearningai\n               'UCAWsBMQY4KSuOuGODki-l7A', # @emma_ding\n               'UCJublDh2UsiIKsAE1553miw', # @GregHogg\n               'UCSNeZleDn9c74yQc-EKnVTA', # @kaggle\n               'UCq6XkhO5SZ66N04IcPbqNcw', # @KeithGalli\n               'UCfzlCWGWYyIQ0aLC5w48gBQ', # @sentdex\n               'UCV8e2g4IWQqK71bbzGDEI4Q', # @DataProfessor\n               'UCbfYPyITQ-7l4upoX8nvctg', # @TwoMinutePapers\n               'UC2UXDak6o7rBm23k3Vv5dww', # @TinaHuang1\n               'UC79Gv3mYp6zKiSwYemEik9A', # @DataCamp \n               'UCzL_0nIe8B4-7ShhVPfJkgw', # @Datasciencedojo\n               'UCnVzApLJE2ljPZSeQylSEyg', # @dataschool\n               'UCFrjdcImgcQVyFbK04MBEhA', # @BrandonFoltz\n               'UCLLw7jmFsvfIVaUFsLs8mlQ', # @LukeBarousse\n               'UCBPRJjIWfyNG4X-CRbnv78A', # @abhishekkrthakur\n               'UCEBpSZhI1X8WaP-kY_2LLcg', # @365DataScience\n               'UCtYLUTtgS3k1Fg4y5tAhLbw', # @statquest\n               'UC4JX40jDee_tINbkjycV4Sg', # @TechWithTim\n               'UCQSMCVUX1HgrwxJhO_7VrJQ', # @DataInterview\n               'UCDvErgK0j5ur3aLgn6U-LqQ', # @DataTalksClub\n               'UCYQS3jkfB79Diyr9sQJAj5Q', # @landingai\n               'UCBqMxo5ch1NN0hZsM58gf6Q', # @ReuvenLerner\n               'UC7kjWIK1H8tfmFlzZO-wHMw', # @twimlai\n               'UCBpxspUNl1Th33XbugiHJzw', # @MITCSAIL\n               'UCHB9VepY6kYvZjj0Bgxnpbw', # @connorshorten6311\n               'UCEqgmyWChwvt6MFGGlmUQCQ', # @allenai\n               'UCqrILQNl5Ed9Dz6CGMyvMTQ', # @CleverProgrammer\n               'UCrBzGHKmGDcwLFnQGHJ3XYg', # @giantneuralnetwork\n               'UCY8mzqqGwl5_bTpBY9qLMAA', # @andreaskayy\n               'UC12LqyqTQYbXatYS9AA7Nuw', # @machinelearninguniversity1942\n               'UCHlNU7kIZhRgSbhHvFoy72w', # @HuggingFace\n               'UCJS9pqu9BzkAMNTmzNMNhvg', # @googlecloudtech\n               'UCXZCJLdBC09xxGZ6gcdrc6A', # @OpenAI\n              ]\napi_service_name = \"youtube\"\napi_version = \"v3\"\n\n# Get credentials and create an API client\nyoutube = build(\n    api_service_name, api_version, developerKey=api_key)","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"kudos to @Thuvu5 for the neat functions. saved me a lot of time! 😁","metadata":{}},{"cell_type":"code","source":"def get_channel_stats(youtube, channel_ids):\n    \n    \"\"\"\n    Get channel stats\n    \n    Params:\n    ------\n    youtube: build object of Youtube API\n    channel_ids: list of channel IDs\n    \n    Returns:\n    ------\n    dataframe with all channel stats for each channel ID\n    \n    Reference:\n    https://developers.google.com/youtube/v3/docs/channels#resource\n    \n    \"\"\"\n    \n    all_data = []\n    \n    request = youtube.channels().list(\n        part=\"snippet,contentDetails,statistics\",\n        id=','.join(channel_ids)\n    )\n    response = request.execute()\n\n    # loop through items\n    for item in response['items']:\n        data = {'channelName': item['snippet']['title'],\n                'handle': item['snippet']['customUrl'],\n                'created': item['snippet']['publishedAt'],\n                'subscribers': item['statistics']['subscriberCount'],\n                'views': item['statistics']['viewCount'],\n                'totalVideos': item['statistics']['videoCount'],\n                'playlistId': item['contentDetails']['relatedPlaylists']['uploads']\n        }\n        \n        all_data.append(data)\n        \n    return pd.DataFrame(all_data)\n\ndef get_video_ids(youtube, playlist_id):\n    \n    video_ids = []\n    \n    request = youtube.playlistItems().list(\n        part=\"snippet,contentDetails\",\n        playlistId=playlist_id,\n        maxResults = 50\n    )\n    response = request.execute()\n    \n    for item in response['items']:\n        video_ids.append(item['contentDetails']['videoId'])\n        \n    next_page_token = response.get('nextPageToken')\n    while next_page_token is not None:\n        request = youtube.playlistItems().list(\n                    part='contentDetails',\n                    playlistId = playlist_id,\n                    maxResults = 50,\n                    pageToken = next_page_token)\n        response = request.execute()\n\n        for item in response['items']:\n            video_ids.append(item['contentDetails']['videoId'])\n\n        next_page_token = response.get('nextPageToken')\n        \n    return video_ids\n    \n    \ndef get_video_details(youtube, video_ids):\n\n    all_video_info = []\n    \n    for i in range(0, len(video_ids), 50):\n        request = youtube.videos().list(\n            part=\"snippet,contentDetails,statistics\",\n            id=','.join(video_ids[i:i+50])\n        )\n        response = request.execute() \n\n        for video in response['items']:\n            stats_to_keep = {'snippet': ['channelTitle', 'title', 'description', 'tags', 'publishedAt'],\n                             'statistics': ['viewCount', 'likeCount', 'favouriteCount', 'commentCount'],\n                             'contentDetails': ['duration', 'definition', 'caption']\n                            }\n            video_info = {}\n            video_info['video_id'] = video['id']\n\n            for k in stats_to_keep.keys():\n                for v in stats_to_keep[k]:\n                    try:\n                        video_info[v] = video[k][v]\n                    except:\n                        video_info[v] = None\n\n            all_video_info.append(video_info)\n    \n    return pd.DataFrame(all_video_info)","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## List of my top DS YouTube channels 🏆","metadata":{}},{"cell_type":"code","source":"channel_stats = get_channel_stats(youtube, channel_ids)\nchannel_stats[['subscribers','views','totalVideos']] = \\\n    channel_stats[['subscribers','views','totalVideos']].astype(int)\nchannel_stats = channel_stats.sort_values(by=\"subscribers\", ascending=False).reset_index(drop=True)\nchannel_stats.loc[:, \"subscribers\"] =channel_stats[\"subscribers\"].map('{:,.0f}'.format)\nchannel_stats.loc[:, \"views\"] =channel_stats[\"views\"].map('{:,.0f}'.format)\nchannel_stats['created'] = pd.to_datetime(channel_stats['created'].astype(str).str[:10]).dt.date\nscrolling(channel_stats)","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 8))\nchannel_stats['subscribers'] = channel_stats['subscribers'].astype(str).str.replace(\",\",\"\").astype(int)\nsns.barplot(x='subscribers', y='channelName', data=channel_stats, orient='h')\nplt.title('Subscribers by Channel')\nplt.xlabel('Subscribers')\nplt.ylabel('Channel Name')\nplt.show()","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 8))\nchannel_stats['views'] = channel_stats['views'].astype(str).str.replace(\",\",\"\").astype(int)\nchannel_stats = channel_stats.sort_values(by=\"views\", ascending=False)\nsns.barplot(x='views', y='channelName', data=channel_stats, orient='h')\nplt.title('Views by Channel')\nplt.xlabel('Views')\nplt.ylabel('Channel Name')\nplt.show()","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 8))\nchannel_stats['totalVideos'] = channel_stats['totalVideos'].astype(str).str.replace(\",\",\"\").astype(int)\nchannel_stats = channel_stats.sort_values(by=\"totalVideos\", ascending=False)\nsns.barplot(x='totalVideos', y='channelName', data=channel_stats, orient='h')\nplt.title('Total Number of Videos by Channel')\nplt.xlabel('Total Number of Videos')\nplt.ylabel('Channel Name')\nplt.show()","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Further data scraping using YouTube API","metadata":{}},{"cell_type":"code","source":"%%time\n# Get video IDs\nvideo_ids = []\nfor id in channel_stats['playlistId']:\n    video_ids += get_video_ids(youtube, id)\nlen(video_ids)","metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n# Get video details\nvideo_df = get_video_details(youtube, video_ids[0:]).drop(columns=['favouriteCount','definition','caption','duration'])\nvideo_df.head(3)","metadata":{"_kg_hide-input":false,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"video_df.shape","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Some data preprocessing...","metadata":{}},{"cell_type":"code","source":"# ignore the \"tweets\" lol. copy + paste from my twitter sentiment analysis notebook haha\n\ndef preprocess_word(word):\n    # Remove punctuation\n    word = word.strip('\\'\"?!,.()[]:;')\n    \n    # Convert more than 2 letter repetitions to 2 letter\n    # funnnnny --> funny\n    word = re.sub(r'(.)\\1+', r'\\1\\1', word)\n    # Remove - & '\n    word = re.sub(r'(-|\\')', '', word)\n    return word\n\n\ndef is_valid_word(word):\n    # Check if word begins with an alphabet\n    return (re.search(r'^[a-zA-Z][a-z0-9A-Z\\._]*$', word) is not None)\n\n\ndef handle_emojis(tweet):\n    # Smile -- :), : ), :-), (:, ( :, (-:, :')\n    tweet = re.sub(r'(:\\s?\\)|:-\\)|\\(\\s?:|\\(-:|:\\'\\))', ' EMO_POS ', tweet)\n    # Laugh -- :D, : D, :-D, xD, x-D, XD, X-D\n    tweet = re.sub(r'(:\\s?D|:-D|x-?D|X-?D)', ' EMO_POS ', tweet)\n    # Love -- <3, :*\n    tweet = re.sub(r'(<3|:\\*)', ' EMO_POS ', tweet)\n    # Wink -- ;-), ;), ;-D, ;D, (;,  (-;\n    tweet = re.sub(r'(;-?\\)|;-?D|\\(-?;)|😉', ' EMO_POS ', tweet)\n    # Sad -- :-(, : (, :(, ):, )-:\n    tweet = re.sub(r'(:\\s?\\(|:-\\(|\\)\\s?:|\\)-:)', ' EMO_NEG ', tweet)\n    # Cry -- :,(, :'(, :\"(\n    tweet = re.sub(r'(:,\\(|:\\'\\(|:\"\\()', ' EMO_NEG ', tweet)\n    return tweet\n\n\ndef remove_emoji(tweet):\n    return emoji.replace_emoji(tweet, replace=\" \")\n\n\ndef preprocess_sentence(tweet):\n    processed_tweet = []\n    # Convert to lower case\n    tweet = tweet.lower()\n    \n    #Clean only digits\n    tweet = re.sub(\"^\\d+\\s|\\s\\d+\\s|\\s\\d+$\", \" \", tweet)\n    \n    # Replaces URLs with the word URL\n    #tweet = re.sub(r'((www\\.[\\S]+)|(https?://[\\S]+))', ' URL ', tweet)\n    tweet = re.sub(r'((www\\.[\\S]+)|(https?://[\\S]+))', '', tweet)\n    \n    # Replace @handle with the word USER_MENTION\n    #tweet = re.sub(r'@[\\S]+', 'USER_MENTION', tweet)\n    tweet = re.sub(r'@[\\S]+', '', tweet)\n    \n    # Replaces #hashtag with hashtag\n    #tweet = re.sub(r'#(\\S+)', r' \\1 ', tweet)\n    tweet = re.sub(r'#(\\S+)', '', tweet)\n    \n    # Remove RT (retweet)\n    tweet = re.sub(r'\\brt\\b', '', tweet)\n    \n    # Replace 2+ dots with space\n    tweet = re.sub(r'\\.{2,}', ' ', tweet)\n    \n    # Strip space, \" and ' from tweet\n    tweet = tweet.strip(' \"\\'')\n\n    # Replace emojis with either EMO_POS or EMO_NEG\n    #tweet = handle_emojis(tweet)\n\n    #remove emojis\n    tweet = remove_emoji(tweet)\n    \n    # Replace multiple spaces with a single space\n    tweet = re.sub(r'\\s+', ' ', tweet)\n    words = tweet.split()\n\n    for word in words:\n        word = preprocess_word(word)\n        processed_tweet.append(word)\n\n    return ' '.join(processed_tweet)\n\ndef get_unique_words(string):\n    words = string.split()  # Split the string into a list of words\n    unique_words = set()  # Create an empty set\n    result = []\n\n    for word in words:\n        if word not in unique_words:\n            unique_words.add(word)  # Add word to the set\n            result.append(word)  # Add word to the result list\n\n    return ' '.join(result)","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Clean & transform the dataframe\nvideo_df['publishedAt'] = pd.to_datetime(video_df['publishedAt'].astype(str).str[:10]).dt.date\nvideo_df['year'] = pd.to_datetime(video_df['publishedAt']).dt.year\nvideo_df['month'] = pd.to_datetime(video_df['publishedAt']).dt.month\nvideo_df['day'] = pd.to_datetime(video_df['publishedAt']).dt.day\nvideo_df['day_of_week'] = pd.to_datetime(video_df['publishedAt']).dt.dayofweek\nvideo_df['year-month'] = video_df['publishedAt'].astype(str).str[:7]\nvideo_df['processed_title'] = video_df['title'].apply(preprocess_sentence)\nvideo_df['processed_desc'] = video_df['description'].apply(preprocess_sentence)\nvideo_df['tags'] = video_df['tags'].astype(str)\nvideo_df['processed_tags'] = video_df['tags'].apply(preprocess_sentence)\nvideo_df['concat_words'] = video_df['processed_title']+video_df['processed_tags']\nvideo_df['concat_word'] = video_df['concat_words'].apply(get_unique_words)\nvideo_df = video_df.sort_values(by=\"publishedAt\", ascending=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"video_df.head(3)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Exploratory Data Analysis\n\nLet's looks at data in the past 10 years.","metadata":{}},{"cell_type":"code","source":"df1 = video_df.copy().groupby(['channelTitle','year','year-month'])['video_id'].count().reset_index()\ndf1 = df1.rename(columns={'video_id':'video_count'})\ndf1['year-month'] = pd.to_datetime(df1['year-month'])\ndf1 = df1[df1['year']>2013]","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objects as go\n\n# Create a figure and define the layout\nfig = go.Figure()\n\n# Iterate over unique channel titles\nfor channel_title in df1['channelTitle'].unique():\n    # Filter data for the current channel title\n    channel_data = df1[df1['channelTitle'] == channel_title]\n    \n    # Extract x and y values\n    x = channel_data['year-month']\n    y = channel_data['video_count']\n    \n    # Add trace for each channel title\n    fig.add_trace(go.Scatter(x=x, y=y, mode='lines', name=channel_title))\n    \n# Update layout\nfig.update_layout(\n    title='Video Count by Channel',\n    xaxis_title='Year-Month',\n    yaxis_title='Video Count',\n    hovermode='x',\n    legend=dict(\n        x=0,\n        y=-0.1,\n        orientation='h',\n        bgcolor='rgba(255, 255, 255, 0.5)'),\n    width=1050, \n    height=800 \n)\n\n# Show the plot\nfig.show()","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Interestingly, from my small population of DS Youtube channels, the number of new channels only started to pick up  in 2017. What heppened then? \n","metadata":{}},{"cell_type":"markdown","source":"# WorldClouds for YouTube Titles and Tags in the Past 10 Years! 🤯\n","metadata":{}},{"cell_type":"code","source":"# some helper functions for wordclouds\n\ndef get_top_words(df, col, num_words, exclude_words):\n    word_counts = Counter()\n    for text in df[col]:\n        if isinstance(text, str):  # Check if text is a valid string\n            words = text.lower().split()\n            words = [word for word in words if word not in exclude_words]\n            word_counts.update(words)\n    top_words = word_counts.most_common(num_words)\n    return top_words\n\n\ndef wordcloudplot(df, col, nos, title):\n    df[col] = df[col].str.replace('[^\\w\\s]', '')\n    stopwords_list = stopwords.words('english')\n    additional_exclude_words = ['like',\"n't\",'use','using','get','one','amp','could','would','im','even',\\\n                                'none','data','science','course','courses','python','machine','learning','learn',\\\n                                'video','best','role','facebook','twitter','subscribe','programming','tutorial',\\\n                                'scientist','scientists','two','minute','papers','paper','paperstwo','coding',\\\n                                'beginners','google','cloud','gcp','analysis']\n    exclude_words = set(stopwords_list + additional_exclude_words)\n    top_words = get_top_words(df, col, nos, exclude_words)\n    wordcloud = WordCloud(width=800, \n                          height=400,\n                          contour_width=3,\n                          contour_color='midnightblue', \n                          background_color=\"white\").generate_from_frequencies(dict(top_words))\n    plt.figure(figsize=(10, 6))\n    plt.imshow(wordcloud, interpolation=\"bilinear\")\n    plt.axis(\"off\")\n    plt.title(title, fontsize=16, pad=20, y=-0.15) \n    plt.show()\n    ","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2023 = video_df[video_df['year']== 2023]\nwordcloudplot(df_2023, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2023\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2022 = video_df[video_df['year']== 2022]\nwordcloudplot(df_2022, \"concat_word\", 500, \"Top Words for YouTube Title in 2022\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2021 = video_df[video_df['year']== 2021]\nwordcloudplot(df_2021, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2021\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2020 = video_df[video_df['year']== 2020]\nwordcloudplot(df_2020, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2020\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2019 = video_df[video_df['year']== 2019]\nwordcloudplot(df_2019, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2019\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2018 = video_df[video_df['year']== 2018]\nwordcloudplot(df_2018, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2018\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2017 = video_df[video_df['year']== 2017]\nwordcloudplot(df_2017, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2017\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2016 = video_df[video_df['year']== 2016]\nwordcloudplot(df_2016, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2016\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2015 = video_df[video_df['year']== 2015]\nwordcloudplot(df_2015, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2015\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_2014 = video_df[video_df['year']== 2014]\nwordcloudplot(df_2014, \"concat_word\", 500, \"Top Words for YouTube Title and Tags in 2014\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I have filtered out some common words like \"data\" and \"science\" so we can see past the haze in the wordclouds. \n\nNot many channels included in my list before 2017 tho so the results are not so interesting.","metadata":{}},{"cell_type":"code","source":"video_df.to_csv(\"video_df.csv\")\ndf1.to_csv(\"df1.csv\")","metadata":{"_kg_hide-input":true,"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# After thoughts\n\nThis was a last minute effort to submit in this competition as I saw the chance to try out the Youtube api and I thought it would be interesting to include all channels I could find and slam it inside this report.\n\nTurns out that the api is kinda unstable in retreiving too many video_ids at once where some get lost on the way so I had to ditch some of those channels with more than 1000 vids. Tried a few times and the total retrieved ids are always different who knows how many ids will turn out in my kernel after I click the saved version. 🤣 \n\n\n\n\n**Version history:**\n- version 2: gave it a try to include all channels that have more than 1000 vids! fingers crossed!    \n- version 3: seems like the api successfully captured more video_ids! also adjusted the wordclouds ban word list\n","metadata":{}}]}