{ "id": 326520, "title": "Additional Supporting Dataset 📊", "url": "/competitions/AI4Code/discussion/326520", "author_name": "", "post_date": "2022-05-22T22:27:02.594371100Z", "votes": 50, "comment_count": 6, "views": 0, "content": "

Hello Everyone
\nI have created an additional dataset from notebooks extracted from the KGTorrent: A Dataset of Python Jupyter Notebooks from Kaggle dataset source.

\n

This dataset consists of a total of notebooks code and markdown cells.
\nLink to the dataset - https://www.kaggle.com/datasets/odins0n/ai4code-custom-data

\n

The original KGTorrent dataset was 75GB+ in size so all the extraction and preprocessing were done on my local PC. Link to processing notebook - Click here

\n

The dataset consists of a single dataframe with the following columns:

\n
Source\nCell Type\nRank\nPct Rank\nNotebook ID\n
\n

Please let me know if you have any feedback 😀

", "messages": [ { "id": "1798325", "postDate": "05/22/2022 22:27:02", "content": "

Hello Everyone
\nI have created an additional dataset from notebooks extracted from the KGTorrent: A Dataset of Python Jupyter Notebooks from Kaggle dataset source.

\n

This dataset consists of a total of notebooks code and markdown cells.
\nLink to the dataset - https://www.kaggle.com/datasets/odins0n/ai4code-custom-data

\n

The original KGTorrent dataset was 75GB+ in size so all the extraction and preprocessing were done on my local PC. Link to processing notebook - Click here

\n

The dataset consists of a single dataframe with the following columns:

\n
Source\nCell Type\nRank\nPct Rank\nNotebook ID\n
\n

Please let me know if you have any feedback 😀

", "rawMarkdown": "Hello Everyone\nI have created an additional dataset from notebooks extracted from the [KGTorrent: A Dataset of Python Jupyter Notebooks from Kaggle](https://arxiv.org/abs/2103.10558) dataset source. \n\nThis dataset consists of a total of 168787 notebooks code and markdown cells.\nLink to the dataset - https://www.kaggle.com/datasets/odins0n/ai4code-custom-data \n\nThe original KGTorrent dataset was 75GB+ in size so all the extraction and preprocessing were done on my local PC. Link to processing notebook - [Click here](https://github.com/sanskar-hasija/kaggle/blob/main/notebooks/AI4Code-Custom%20Data.ipynb) \n\nThe dataset consists of a single dataframe with the following columns:\n```\nSource\nCell Type\nRank\nPct Rank\nNotebook ID\n```\nPlease let me know if you have any feedback 😀", "votes": null }, { "id": "1798386", "postDate": "05/23/2022 01:38:16", "content": "

Hey @odins0n best of luck.You are doing great.

", "rawMarkdown": "Hey @odins0n best of luck.You are doing great.", "votes": null }, { "id": "1798603", "postDate": "05/23/2022 07:36:47", "content": "

Thanks @ramjasmaurya

", "rawMarkdown": "Thanks @ramjasmaurya", "votes": null }, { "id": "1859838", "postDate": "07/18/2022 02:17:20", "content": "

Thanks for the resource @odins0n

", "rawMarkdown": "Thanks for the resource @odins0n", "votes": null }, { "id": "1880179", "postDate": "08/01/2022 14:13:08", "content": "

Have somebody figured out if we are allowed to use this dataset by the rules?

\n

I saw similar discussions on other topics (e.g. this), and question is basically about licenses used in notebooks. But not sure how to check if notebooks in this dataset were filtered based on the permissive license or not.

", "rawMarkdown": "Have somebody figured out if we are allowed to use this dataset by the rules? \n\nI saw similar discussions on other topics (e.g. [this](https://www.kaggle.com/competitions/AI4Code/discussion/340842)), and question is basically about licenses used in notebooks. But not sure how to check if notebooks in this dataset were filtered based on the permissive license or not.", "votes": null }, { "id": "1885682", "postDate": "08/05/2022 10:22:29", "content": "

This dataset has collected with permissive licenses. There's about 800k notebooks, but not from Kaggle.

", "rawMarkdown": "This [dataset](https://zenodo.org/record/6555157#.YuzvAy96BpQ) has collected with permissive licenses. There's about 800k notebooks, but not from Kaggle.", "votes": null }, { "id": "1885789", "postDate": "08/05/2022 11:32:03", "content": "

Any public kaggle notebooks must be licensed under Apache 2.0 License.

", "rawMarkdown": "Any public kaggle notebooks must be licensed under Apache 2.0 License.", "votes": null } ], "comments": [ { "id": 1798386, "author_name": "ramjasmaurya", "author_url": "", "post_date": "05/23/2022 01:38:16", "content": "

Hey @odins0n best of luck.You are doing great.

", "votes": null, "replies": [ { "id": 1798603, "author_name": "odins0n", "author_url": "", "post_date": "05/23/2022 07:36:47", "content": "

Thanks @ramjasmaurya

", "votes": null, "replies": [] } ] }, { "id": 1859838, "author_name": "apurbapandey", "author_url": "", "post_date": "07/18/2022 02:17:20", "content": "

Thanks for the resource @odins0n

", "votes": null, "replies": [] }, { "id": 1880179, "author_name": "bminaiev", "author_url": "", "post_date": "08/01/2022 14:13:08", "content": "

Have somebody figured out if we are allowed to use this dataset by the rules?

\n

I saw similar discussions on other topics (e.g. this), and question is basically about licenses used in notebooks. But not sure how to check if notebooks in this dataset were filtered based on the permissive license or not.

", "votes": null, "replies": [ { "id": 1885682, "author_name": "konstantgr", "author_url": "", "post_date": "08/05/2022 10:22:29", "content": "

This dataset has collected with permissive licenses. There's about 800k notebooks, but not from Kaggle.

", "votes": null, "replies": [] }, { "id": 1885789, "author_name": "yufuin", "author_url": "", "post_date": "08/05/2022 11:32:03", "content": "

Any public kaggle notebooks must be licensed under Apache 2.0 License.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1798325": "Hello Everyone\nI have created an additional dataset from notebooks extracted from the [KGTorrent: A Dataset of Python Jupyter Notebooks from Kaggle](https://arxiv.org/abs/2103.10558) dataset source. \n\nThis dataset consists of a total of 168787 notebooks code and markdown cells.\nLink to the dataset - https://www.kaggle.com/datasets/odins0n/ai4code-custom-data \n\nThe original KGTorrent dataset was 75GB+ in size so all the extraction and preprocessing were done on my local PC. Link to processing notebook - [Click here](https://github.com/sanskar-hasija/kaggle/blob/main/notebooks/AI4Code-Custom%20Data.ipynb) \n\nThe dataset consists of a single dataframe with the following columns:\n```\nSource\nCell Type\nRank\nPct Rank\nNotebook ID\n```\nPlease let me know if you have any feedback 😀", "1798386": "Hey @odins0n best of luck.You are doing great.", "1798603": "Thanks @ramjasmaurya", "1859838": "Thanks for the resource @odins0n", "1880179": "Have somebody figured out if we are allowed to use this dataset by the rules? \n\nI saw similar discussions on other topics (e.g. [this](https://www.kaggle.com/competitions/AI4Code/discussion/340842)), and question is basically about licenses used in notebooks. But not sure how to check if notebooks in this dataset were filtered based on the permissive license or not.", "1885682": "This [dataset](https://zenodo.org/record/6555157#.YuzvAy96BpQ) has collected with permissive licenses. There's about 800k notebooks, but not from Kaggle.", "1885789": "Any public kaggle notebooks must be licensed under Apache 2.0 License." }, "source": "meta" }