Hi Kagglers, I was looking into joining this competition a little late and I noticed interestingly enough there are virtually 0 notebooks with cross validation? I have seen in discussion that competition organizers suggest the goal is not to do these predictions in real time.
\n\nI would think to see more notebooks utilizing validation of some sort so I just was curious if I was missing something?
", "messages": [ { "id": 2484533, "postDate": "2023-10-16T14:51:41.023Z", "content": "Hi Kagglers, I was looking into joining this competition a little late and I noticed interestingly enough there are virtually 0 notebooks with cross validation? I have seen in discussion that competition organizers suggest the goal is not to do these predictions in real time.
\n\nI would think to see more notebooks utilizing validation of some sort so I just was curious if I was missing something?
", "rawMarkdown": "Hi Kagglers, I was looking into joining this competition a little late and I noticed interestingly enough there are virtually 0 notebooks with cross validation? I have seen in discussion that competition organizers suggest the goal is not to do these predictions in real time. \n\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/437534#2427761\n\nI would think to see more notebooks utilizing validation of some sort so I just was curious if I was missing something? ", "votes": 6 }, { "id": 2485103, "postDate": "2023-10-17T00:18:06.287Z", "content": "A 5 fold CV, for example, can be expected to take up till 2-3 times the duration (if you’re using the lightweight dataset for CV) and 6 times (5 folds + 1 inference) if the entire train dataset is used. It would be too slow to run in 9 hours
", "rawMarkdown": "A 5 fold CV, for example, can be expected to take up till 2-3 times the duration (if you’re using the lightweight dataset for CV) and 6 times (5 folds + 1 inference) if the entire train dataset is used. It would be too slow to run in 9 hours", "votes": 4, "replies": [ { "id": 2485121, "postDate": "2023-10-17T00:34:59.073Z", "content": "Yep absolutely, I think if someone wanted to go this route they would need to figure out creative ways to train and they would definitely need to make separate train/inference notebooks. Could be helpful to move off Kaggle resources too depending on the strategy.
", "rawMarkdown": "Yep absolutely, I think if someone wanted to go this route they would need to figure out creative ways to train and they would definitely need to make separate train/inference notebooks. Could be helpful to move off Kaggle resources too depending on the strategy.", "votes": 1 }, { "id": 2486434, "postDate": "2023-10-17T23:08:45.573Z", "content": "Agreed. The training set is already smoking my system, haha.
", "rawMarkdown": "Agreed. The training set is already smoking my system, haha." } ] }, { "id": 2484539, "postDate": "2023-10-16T14:57:34.320Z", "content": "I've previously asked a similar question, in this discussion.
\nI think the answer is just that:
A 5 fold CV, for example, can be expected to take up till 2-3 times the duration (if you’re using the lightweight dataset for CV) and 6 times (5 folds + 1 inference) if the entire train dataset is used. It would be too slow to run in 9 hours
", "votes": 4, "replies": [ { "id": 2485121, "author_name": "Cody_Null", "author_url": "", "post_date": "2023-10-17T00:34:59.073000", "content": "Yep absolutely, I think if someone wanted to go this route they would need to figure out creative ways to train and they would definitely need to make separate train/inference notebooks. Could be helpful to move off Kaggle resources too depending on the strategy.
", "votes": 1, "replies": [] }, { "id": 2486434, "author_name": "ejwlk", "author_url": "", "post_date": "2023-10-17T23:08:45.573000", "content": "Agreed. The training set is already smoking my system, haha.
", "votes": 0, "replies": [] } ] }, { "id": 2484539, "author_name": "Yuchen20", "author_url": "", "post_date": "2023-10-16T14:57:34.320000", "content": "I've previously asked a similar question, in this discussion.
\nI think the answer is just that: