I hope I can explain this well, here I try:
\nOn timeline page it is said:
\n'Starting after the final submission deadline there will be periodic updates to the leaderboard to reflect market data updates that will be run against selected notebooks. Updates will take place roughly every two weeks.'
Can I interpret above paragraph as:
\nThe test data will be provided in 'two weeks batches', and the data until 'test_time' will be filled with supplemental_train.csv, so that all ground truth targets are available until that specific 'two weeks batch'.
What is really meant by saying 'run against selected notebooks'? If same test data is not run for all models, how will the comparison be made among models? i.e.: what if my perfect model will have to predict a week starting with an Elon Musk tweet on cryptos? :-)
I hope I can explain this well, here I try:
\nOn timeline page it is said:
\n'Starting after the final submission deadline there will be periodic updates to the leaderboard to reflect market data updates that will be run against selected notebooks. Updates will take place roughly every two weeks.'
Can I interpret above paragraph as:
\nThe test data will be provided in 'two weeks batches', and the data until 'test_time' will be filled with supplemental_train.csv, so that all ground truth targets are available until that specific 'two weeks batch'.
What is really meant by saying 'run against selected notebooks'? If same test data is not run for all models, how will the comparison be made among models? i.e.: what if my perfect model will have to predict a week starting with an Elon Musk tweet on cryptos? :-)
\n\n'run against selected notebooks'
\n
This means that they will evaluate/score -periodically every 2 weeks - only the final two selected submissions from each team (for e.g. this was also the case with the recent Jane street competition)
\nThe important point i'd like also a clarification is if the market data updates are cumulative, i.e. the score/data of Update-Week-N+1 contains also data from Week-N
", "rawMarkdown": "> 'run against selected notebooks'\n\nThis means that they will evaluate/score -periodically every 2 weeks - only the final two selected submissions from each team (for e.g. this was also the case with the recent Jane street competition) \n\nThe important point i'd like also a clarification is if the market data updates are cumulative, i.e. the score/data of Update-Week-N+1 contains also data from Week-N", "votes": 1, "replies": [ { "id": 1587260, "postDate": "2021-11-18T15:38:08.200Z", "content": "I was thinking, that it was emphasizing a second selection among all 'selected two notebooks', rather than referring directly to the 'selected two notebooks',
\n…now I see.
@abdulkadirguner Under the Data ribbon on the competition overview under the Time-Series API Details it says:
\n\n\nExpect to see roughly three months worth of data in the test set. Until the forecasting phase of the competition, the API will just deliver a slice of the training data.
\n
Do you mean you interpret this as, test_df which is in the iter_test function will have roughly 3 months worth of historic datA?
Here is my understanding:
\nHey @abdulkadirguner - That makes perfect sense! One final clarification would mean a real lot to me. In your part 4 of your answer you said:
\n\n\nIn that next iteration (from item3 above), all the ground truth data until that specific test window will be available to your model.
\n
For every iteration of that iter_test, I'll have the new chunk of data (which is test_df that is in chunks of 2 weeks) but mainly what I'm asking is the ground truth data from 2018 up until the current period; Can I assume that will be in the specific test_df in every iteration? or when I write my code, I will read in that supplemental_train csv and then concatenate the new data which is test_df? That part mainly confused me. I ask this because I will need to understand so I know how to write the last bit of code to make predictions.
Just for reference, in the data ribbon section under Files, it says:
\n\nsupplemental_train.csv - After the submission period is over this file's data will be replaced with cryptoasset prices from the submission period. In the Evaluation phase, the train, train supplement, and test set will be contiguous in time, apart from any missing data. The current copy, which is just filled approximately the right amount of data from train.csv is provided as a placeholder.
\n
From this, I assumed that in the actual testing (Evaluation) phase, the supplemental_train csv will be available for me to read in which will have all the data from 2018, then use to make inferences ALONG with the test_df which will have the newest chunk of data.
Sorry if my question is a bit specific and incoherent - this is my first Kaggle competition.
\nThank you for your help!
", "rawMarkdown": "Hey @abdulkadirguner - That makes perfect sense! One final clarification would mean a real lot to me. In your part 4 of your answer you said: \n> In that next iteration (from item3 above), all the ground truth data until that specific test window will be available to your model.\n\nFor every iteration of that `iter_test`, I'll have the new chunk of data (which is `test_df` that is in chunks of 2 weeks) but mainly what I'm asking is the ground truth data from 2018 up until the current period; Can I assume that will be in the specific `test_df` in every iteration? or when I write my code, I will read in that `supplemental_train` csv and then concatenate the new data which is `test_df`? That part mainly confused me. I ask this because I will need to understand so I know how to write the last bit of code to make predictions.\n\nJust for reference, in the data ribbon section under `Files`, it says:\n> supplemental_train.csv - After the submission period is over this file's data will be replaced with cryptoasset prices from the submission period. In the Evaluation phase, the train, train supplement, and test set will be contiguous in time, apart from any missing data. The current copy, which is just filled approximately the right amount of data from train.csv is provided as a placeholder.\n\nFrom this, I assumed that in the actual testing (Evaluation) phase, the `supplemental_train` csv will be available for me to read in which will have all the data from 2018, then use to make inferences ALONG with the `test_df` which will have the newest chunk of data.\n\nSorry if my question is a bit specific and incoherent - this is my first Kaggle competition. \n\nThank you for your help!" }, { "id": 1591185, "postDate": "2021-11-22T04:27:20.363Z", "content": "The evaluation will be per two weeks period, but I am not sure about how this test will be implemented. Will the 'two weeks window' be partioned into smaller time windows such as daily test chunks, so that your model will have to predict 'just one day' and will have everything upto that one day available?,, I am not sure. (assuming: model assessment window==one day)
\nBut my assumption/understanding is, per one specific window, supplemental_train will include upto the start of that specific test_time window only, in other words supplemental_data will not be provided for each and every single iteration but per each 'model assessment window'.
I am new to this competition, so this assumption needs confirmation.
", "rawMarkdown": "The evaluation will be per two weeks period, but I am not sure about how this test will be implemented. Will the 'two weeks window' be partioned into smaller time windows such as daily test chunks, so that your model will have to predict 'just one day' and will have everything upto that one day available?,, I am not sure. (assuming: model assessment window==one day)\nBut my assumption/understanding is, per one specific window, supplemental_train will include upto the start of that specific test_time window only, in other words supplemental_data will not be provided for each and every single iteration but per each 'model assessment window'. \n\nI am new to this competition, so this assumption needs confirmation. ", "votes": 1 } ] }, { "id": 1587742, "postDate": "2021-11-18T22:57:45.077Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1587247, "author_name": "Ioannis M", "author_url": "", "post_date": "2021-11-18T15:24:57.767000", "content": "\n\n'run against selected notebooks'
\n
This means that they will evaluate/score -periodically every 2 weeks - only the final two selected submissions from each team (for e.g. this was also the case with the recent Jane street competition)
\nThe important point i'd like also a clarification is if the market data updates are cumulative, i.e. the score/data of Update-Week-N+1 contains also data from Week-N
", "votes": 1, "replies": [ { "id": 1587260, "author_name": "GUNER", "author_url": "", "post_date": "2021-11-18T15:38:08.200000", "content": "I was thinking, that it was emphasizing a second selection among all 'selected two notebooks', rather than referring directly to the 'selected two notebooks',
\n…now I see.
@abdulkadirguner Under the Data ribbon on the competition overview under the Time-Series API Details it says:
\n\n\nExpect to see roughly three months worth of data in the test set. Until the forecasting phase of the competition, the API will just deliver a slice of the training data.
\n
Do you mean you interpret this as, test_df which is in the iter_test function will have roughly 3 months worth of historic datA?
Here is my understanding:
\nHey @abdulkadirguner - That makes perfect sense! One final clarification would mean a real lot to me. In your part 4 of your answer you said:
\n\n\nIn that next iteration (from item3 above), all the ground truth data until that specific test window will be available to your model.
\n
For every iteration of that iter_test, I'll have the new chunk of data (which is test_df that is in chunks of 2 weeks) but mainly what I'm asking is the ground truth data from 2018 up until the current period; Can I assume that will be in the specific test_df in every iteration? or when I write my code, I will read in that supplemental_train csv and then concatenate the new data which is test_df? That part mainly confused me. I ask this because I will need to understand so I know how to write the last bit of code to make predictions.
Just for reference, in the data ribbon section under Files, it says:
\n\nsupplemental_train.csv - After the submission period is over this file's data will be replaced with cryptoasset prices from the submission period. In the Evaluation phase, the train, train supplement, and test set will be contiguous in time, apart from any missing data. The current copy, which is just filled approximately the right amount of data from train.csv is provided as a placeholder.
\n
From this, I assumed that in the actual testing (Evaluation) phase, the supplemental_train csv will be available for me to read in which will have all the data from 2018, then use to make inferences ALONG with the test_df which will have the newest chunk of data.
Sorry if my question is a bit specific and incoherent - this is my first Kaggle competition.
\nThank you for your help!
", "votes": 0, "replies": [] }, { "id": 1591185, "author_name": "GUNER", "author_url": "", "post_date": "2021-11-22T04:27:20.363000", "content": "The evaluation will be per two weeks period, but I am not sure about how this test will be implemented. Will the 'two weeks window' be partioned into smaller time windows such as daily test chunks, so that your model will have to predict 'just one day' and will have everything upto that one day available?,, I am not sure. (assuming: model assessment window==one day)
\nBut my assumption/understanding is, per one specific window, supplemental_train will include upto the start of that specific test_time window only, in other words supplemental_data will not be provided for each and every single iteration but per each 'model assessment window'.
I am new to this competition, so this assumption needs confirmation.
", "votes": 1, "replies": [] } ] }, { "id": 1587742, "author_name": "", "author_url": "", "post_date": "2021-11-18T22:57:45.077000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1587173": "I hope I can explain this well, here I try:\nOn timeline page it is said:\n'*Starting after the final submission deadline there will be periodic updates to the leaderboard to reflect market data updates that will be run against selected notebooks. Updates will take place roughly every two weeks.*'\n\n1. Can I interpret above paragraph as:\nThe test data will be provided in **'two weeks batches',** and the data until 'test_time' will be filled with supplemental_train.csv, so that all ground truth targets are available **until that specific 'two weeks batch'**.\n\n2. What is really meant by saying **'run against selected notebooks'?** If **same** test data is not run for **all** models, how will the comparison be made among models? i.e.: *what if my perfect model will have to predict a week starting with an Elon Musk tweet on cryptos? :-)*", "1587247": "> 'run against selected notebooks'\n\nThis means that they will evaluate/score -periodically every 2 weeks - only the final two selected submissions from each team (for e.g. this was also the case with the recent Jane street competition) \n\nThe important point i'd like also a clarification is if the market data updates are cumulative, i.e. the score/data of Update-Week-N+1 contains also data from Week-N", "1590093": "@abdulkadirguner Under the Data ribbon on the competition overview under the Time-Series API Details it says:\n\n> Expect to see roughly three months worth of data in the test set. Until the forecasting phase of the competition, the API will just deliver a slice of the training data.\n\nDo you mean you interpret this as, `test_df` which is in the `iter_test` function will have roughly 3 months worth of historic datA?", "1587742": "" } }