Actually, I can't understand this problem as a beginner.
\nI believe that the deep learning is kind of probabilistic model.
\nIn this problem, how can we induce deep learning models to find relation mapping,
\nnot clusters?
The given data is not independent.
\nIf the data are not independent,
\nI have no idea how to apply deep learning in this problem,
\nIt seems important to organize the training data well.
I think deep learning will still work. However this comp will require some heavy text processing which is a breeze of fresh air, I'm tired of feeding BERT raw data and hoping it will figure everything out by itself.
", "rawMarkdown": "I think deep learning will still work. However this comp will require some heavy text processing which is a breeze of fresh air, I'm tired of feeding BERT raw data and hoping it will figure everything out by itself.", "votes": 5, "replies": [ { "id": 1797738, "postDate": "2022-05-22T09:54:39.380Z", "content": "it seems you found an efficient way. I am still struggling to break 0.79 in CV. This point makes this comp more outstanding and more interesting than other NLP comps, IMO.
", "rawMarkdown": "it seems you found an efficient way. I am still struggling to break 0.79 in CV. This point makes this comp more outstanding and more interesting than other NLP comps, IMO. " }, { "id": 1797741, "postDate": "2022-05-22T09:57:37.690Z", "content": "I agree.. Coleridge was interesting in this regard and this one too.
", "rawMarkdown": "I agree.. Coleridge was interesting in this regard and this one too." } ] }, { "id": 1795523, "postDate": "2022-05-19T21:18:56.053Z", "content": "I think it is a deep learning problem, otherwise, I don't see any other ways to create features and make inference for unseen notebooks. It is true that the data is not independent given each cell, but it is independent given notebooks. By comparison, I think it is similar to the longitudinal data type. Most of the current public notebooks ignore this information but still have good results. It is a deep learning problem, but data needs to be organized well, I believe.
", "rawMarkdown": "I think it is a deep learning problem, otherwise, I don't see any other ways to create features and make inference for unseen notebooks. It is true that the data is not independent given each cell, but it is independent given notebooks. By comparison, I think it is similar to the longitudinal data type. Most of the current public notebooks ignore this information but still have good results. It is a deep learning problem, but data needs to be organized well, I believe.", "votes": 3 }, { "id": 1795362, "postDate": "2022-05-19T17:32:04.047Z", "content": "Actually, I can't understand this problem as a beginner.
\nI believe that the deep learning is kind of probabilistic model.
\nIn this problem, how can we induce deep learning models to find relation mapping,
\nnot clusters?
The given data is not independent.
\nIf the data are not independent,
\nI have no idea how to apply deep learning in this problem,
\nIt seems important to organize the training data well.
Before thinking about deep learning vs non-deep learning, I think it is useful to ask, \"how can we use machine learning to solve this problem?\"
\nThe example notebook that Google provided (here) creates a regression task. For each cell in our (shuffled) notebook, we try to predict the rank of that cell given features of the cell (mainly the text). We then sort the cells by our predicted rank. The idea is that certain text tends to appear at the beginning of a notebook (like \"Introduction\") and certain text at the end (like \"in Conclusion\"). That's probably a good notebook to start with.
\nAnother approach would be to create a pairwise dataset of (cell, other_cell_in_notebook, label) where label == 1 if the other cell directly follows the first cell, else 0. This lets us train a classification model like p(cell A follows cell B) given the text of both cells.
\nEither of these approaches can use deep learning methods or other machine learning algorithms.
", "rawMarkdown": "Before thinking about deep learning vs non-deep learning, I think it is useful to ask, \"how can we use machine learning to solve this problem?\"\n\nThe example notebook that Google provided ([here](https://www.kaggle.com/code/ryanholbrook/getting-started-with-ai4code)) creates a regression task. For each cell in our (shuffled) notebook, we try to predict the rank of that cell given features of the cell (mainly the text). We then sort the cells by our predicted rank. The idea is that certain text tends to appear at the beginning of a notebook (like \"Introduction\") and certain text at the end (like \"in Conclusion\"). That's probably a good notebook to start with.\n\nAnother approach would be to create a pairwise dataset of (cell, other_cell_in_notebook, label) where *label == 1* if the other cell directly follows the first cell, else 0. This lets us train a classification model like p(cell A follows cell B) given the text of both cells.\n\nEither of these approaches can use deep learning methods *or* other machine learning algorithms.", "votes": 1 }, { "id": 1797948, "postDate": "2022-05-22T15:02:26.807Z", "content": "I think the ranking task will help to partially solve this problem. And this problem can be solved by different methods (including using DL)
", "rawMarkdown": "I think the [**ranking** task](https://en.wikipedia.org/wiki/Learning_to_rank) will help to **partially** solve this problem. And this problem can be solved by different methods (including using DL)" }, { "id": 1796371, "postDate": "2022-05-20T18:51:19.310Z", "content": "I think it is a DL problem, though I didn't understand what the baseline did, It seemed that he only use the ordered markdown to do a regression task, which means he train the network to learn something like: sentence that contain \"Preprocessing\" or \"import dependence\" word should have the higher possibility to get a higher score(or smaller number) to be the very front of all markdown, and something like \"Post processing\" may get bigger number when sorting the markdown. But I don't it a \"right way\", and don't think it would work, but it actually work not bad.
\nI think the \"RIGHT WAY\"(or what Google want us to do) is to extract the relationship between \"code\" cell and \"markdown\" cell, make them be pairs of data, and use the code cell (ordered all the time) to judge the right markdown order.
I think deep learning will still work. However this comp will require some heavy text processing which is a breeze of fresh air, I'm tired of feeding BERT raw data and hoping it will figure everything out by itself.
", "votes": 5, "replies": [ { "id": 1797738, "author_name": "Minh Tri Phan", "author_url": "", "post_date": "2022-05-22T09:54:39.380000", "content": "it seems you found an efficient way. I am still struggling to break 0.79 in CV. This point makes this comp more outstanding and more interesting than other NLP comps, IMO.
", "votes": 0, "replies": [] }, { "id": 1797741, "author_name": "Hannes Öhler", "author_url": "", "post_date": "2022-05-22T09:57:37.690000", "content": "I agree.. Coleridge was interesting in this regard and this one too.
", "votes": 0, "replies": [] } ] }, { "id": 1795523, "author_name": "Minh Tri Phan", "author_url": "", "post_date": "2022-05-19T21:18:56.053000", "content": "I think it is a deep learning problem, otherwise, I don't see any other ways to create features and make inference for unseen notebooks. It is true that the data is not independent given each cell, but it is independent given notebooks. By comparison, I think it is similar to the longitudinal data type. Most of the current public notebooks ignore this information but still have good results. It is a deep learning problem, but data needs to be organized well, I believe.
", "votes": 3, "replies": [] }, { "id": 1796611, "author_name": "dricciar", "author_url": "", "post_date": "2022-05-21T01:37:18.480000", "content": "Before thinking about deep learning vs non-deep learning, I think it is useful to ask, \"how can we use machine learning to solve this problem?\"
\nThe example notebook that Google provided (here) creates a regression task. For each cell in our (shuffled) notebook, we try to predict the rank of that cell given features of the cell (mainly the text). We then sort the cells by our predicted rank. The idea is that certain text tends to appear at the beginning of a notebook (like \"Introduction\") and certain text at the end (like \"in Conclusion\"). That's probably a good notebook to start with.
\nAnother approach would be to create a pairwise dataset of (cell, other_cell_in_notebook, label) where label == 1 if the other cell directly follows the first cell, else 0. This lets us train a classification model like p(cell A follows cell B) given the text of both cells.
\nEither of these approaches can use deep learning methods or other machine learning algorithms.
", "votes": 1, "replies": [] }, { "id": 1797948, "author_name": "Pavel Orlov", "author_url": "", "post_date": "2022-05-22T15:02:26.807000", "content": "I think the ranking task will help to partially solve this problem. And this problem can be solved by different methods (including using DL)
", "votes": 0, "replies": [] }, { "id": 1796371, "author_name": "", "author_url": "", "post_date": "2022-05-20T18:51:19.310000", "content": "I think it is a DL problem, though I didn't understand what the baseline did, It seemed that he only use the ordered markdown to do a regression task, which means he train the network to learn something like: sentence that contain \"Preprocessing\" or \"import dependence\" word should have the higher possibility to get a higher score(or smaller number) to be the very front of all markdown, and something like \"Post processing\" may get bigger number when sorting the markdown. But I don't it a \"right way\", and don't think it would work, but it actually work not bad.
\nI think the \"RIGHT WAY\"(or what Google want us to do) is to extract the relationship between \"code\" cell and \"markdown\" cell, make them be pairs of data, and use the code cell (ordered all the time) to judge the right markdown order.