Yes, still one and a half month left till final update. But as it is quite clear that organizers don't plan to
\ndedicate a minute to the issues raised I probably shouldn't waste much more time either.
Before moving on, I thought I would make a quick summary of what I think it has happened in this competition, for what is worth.
\nIt is debatable to what extent what has happened can be considered part of the task of making our models generalize to unseen events/future periods. As @tomforbes has shown it was possible to prepare models for extreme dataset shift and if it was doable then it can be argued that it also can be expected to be done.
\nAs a recap, label of this competition was generated with different weird decisions, one of them critically linked it to the amount of missing values in the data feed. The effect of this was so huge that if data quality significativaly differed of train dataset models were rendered useless. Unless data quality was accounted for at modeling time.
\nIf this is solely what is happening we can not know without data but judging for the private LB results many competitors didn't expect data quality to significatively differ from what was seen during stage 1 as it would have nothing to do with market forces but solely with the host data feed and both host and Kaggle's preprocessing pipeline. At inference time, data quality did differ and many scores are weirdly and unexpectedly low/negative.
\nSo, that regarding the comp. The main point of debate here is to what extent data quality is a legit modeling objective. Who knows, maybe comp host cleverly planned all this as a meta test. But it might be just good old mistakes and lack of attention to label design and data preparation.
\nAs I say, I consider this debatable. As for myself, I wish I had been clever enough to see it coming and hedge it in the way current #1 solution has done. But at the same time in my opinion a competition where generalization is engineered not to happen is a competition of very limited interest for anyone, including the host btw, does the host even care about what Kagglers can do in this domain?
\nLastly, another thing that has happened in this competition is that Kaggle staff has completely ignored legitimate concerns during and after the competition. So many comments/questions ignored. It would have been very easy to say \"data quality is part of the label and data quality of stage2 is poor, but we say that is legit\" or, instead \"we are going to double check data quality\". Or any educated easy answer that showed a minimal concern about doing things right and a minimal respect towards us Kagglers. In this regard I think it's easily the worst organized Kaggle competition I've seen up to date.
\nAnyway, fun as it is to do a forensic data analysis without data it's probably better to stop spending energy on this.
\nI thought this would make a short summary to wrap up and maybe help some other competitors still wondering about all this but yes, time to move on.
", "messages": [ { "id": 1729197, "postDate": "2022-03-19T18:41:53.543Z", "content": "Yes, still one and a half month left till final update. But as it is quite clear that organizers don't plan to
\ndedicate a minute to the issues raised I probably shouldn't waste much more time either.
Before moving on, I thought I would make a quick summary of what I think it has happened in this competition, for what is worth.
\nIt is debatable to what extent what has happened can be considered part of the task of making our models generalize to unseen events/future periods. As @tomforbes has shown it was possible to prepare models for extreme dataset shift and if it was doable then it can be argued that it also can be expected to be done.
\nAs a recap, label of this competition was generated with different weird decisions, one of them critically linked it to the amount of missing values in the data feed. The effect of this was so huge that if data quality significativaly differed of train dataset models were rendered useless. Unless data quality was accounted for at modeling time.
\nIf this is solely what is happening we can not know without data but judging for the private LB results many competitors didn't expect data quality to significatively differ from what was seen during stage 1 as it would have nothing to do with market forces but solely with the host data feed and both host and Kaggle's preprocessing pipeline. At inference time, data quality did differ and many scores are weirdly and unexpectedly low/negative.
\nSo, that regarding the comp. The main point of debate here is to what extent data quality is a legit modeling objective. Who knows, maybe comp host cleverly planned all this as a meta test. But it might be just good old mistakes and lack of attention to label design and data preparation.
\nAs I say, I consider this debatable. As for myself, I wish I had been clever enough to see it coming and hedge it in the way current #1 solution has done. But at the same time in my opinion a competition where generalization is engineered not to happen is a competition of very limited interest for anyone, including the host btw, does the host even care about what Kagglers can do in this domain?
\nLastly, another thing that has happened in this competition is that Kaggle staff has completely ignored legitimate concerns during and after the competition. So many comments/questions ignored. It would have been very easy to say \"data quality is part of the label and data quality of stage2 is poor, but we say that is legit\" or, instead \"we are going to double check data quality\". Or any educated easy answer that showed a minimal concern about doing things right and a minimal respect towards us Kagglers. In this regard I think it's easily the worst organized Kaggle competition I've seen up to date.
\nAnyway, fun as it is to do a forensic data analysis without data it's probably better to stop spending energy on this.
\nI thought this would make a short summary to wrap up and maybe help some other competitors still wondering about all this but yes, time to move on.
", "rawMarkdown": "Yes, still one and a half month left till final update. But as it is quite clear that organizers don't plan to \ndedicate a minute to the issues raised I probably shouldn't waste much more time either.\n\nBefore moving on, I thought I would make a quick summary of what I think it has happened in this competition, for what is worth. \n\nIt is debatable to what extent what has happened can be considered part of the task of making our models generalize to unseen events/future periods. As @tomforbes has shown it was possible to prepare models for extreme dataset shift and if it was doable then it can be argued that it also can be expected to be done.\n\nAs a recap, label of this competition was generated with different weird decisions, one of them critically linked it to the amount of missing values in the data feed. The effect of this was so huge that if data quality significativaly differed of train dataset models were rendered useless. Unless data quality was accounted for at modeling time.\n\nIf this is solely what is happening we can not know without data but judging for the private LB results many competitors didn't expect data quality to significatively differ from what was seen during stage 1 **as it would have nothing to do with market forces but solely with the host data feed and both host and Kaggle's preprocessing pipeline**. At inference time, data quality did differ and many scores are weirdly and unexpectedly low/negative.\n\nSo, that regarding the comp. **The main point of debate here is to what extent data quality is a legit modeling objective**. Who knows, maybe comp host cleverly planned all this as a meta test. But it might be just good old mistakes and lack of attention to label design and data preparation. \n\nAs I say, I consider this debatable. As for myself, I wish I had been clever enough to see it coming and hedge it in the way current #1 solution has done. But at the same time in my opinion a competition where generalization is engineered not to happen is a competition of very limited interest for anyone, including the host btw, does the host even care about what Kagglers can do in this domain? \n\nLastly, another thing that has happened in this competition is that Kaggle staff has completely ignored legitimate concerns during and after the competition. So many comments/questions ignored. It would have been very easy to say \"data quality is part of the label and data quality of stage2 is poor, but we say that is legit\" or, instead \"we are going to double check data quality\". Or any educated easy answer that showed a minimal concern about doing things right and a minimal respect towards us Kagglers. In this regard I think it's easily the worst organized Kaggle competition I've seen up to date.\n\nAnyway, fun as it is to do a forensic data analysis without data it's probably better to stop spending energy on this. \n\nI thought this would make a short summary to wrap up and maybe help some other competitors still wondering about all this but yes, time to move on.", "votes": 15 }, { "id": 1731075, "postDate": "2022-03-22T01:25:35.327Z", "content": "Have you tested your hypothesis with cross validation (CV)? It sounds like you're able to calculate the competition target multiple ways, and you have a model, and you should also have CV setup for the competition. So you could put the three things together and show us what you find.
\nHere's my hypothesis: the effect of any strangeness in the objective function will be insignificant compared to noise.
\nMy model is doing alright (12th place) and I didn't account for any details of the objective function. I just weighted my samples and used L2 cost. My only accounting for missing data was to fill forward.
\nMore on noise:
\nLast update, 6 people in positions 10-20 fell to >400. In addition, a competitor came from position 110 to position 4.
\nBy my estimation, the error in our scores is still about .02, which would imply a couple of things:
\nThe top 50 scores are not very distinguishable. You singled out Tom Forbes, but his model is only doing well now because it did extremely well on the second update. For the third update, it placed around 8th by my estimate, and on the first update, it placed 99th (actual placement). The very high score on the second update was probably mostly luck.
Your current placement is not very distinguishable from 30th place, which would be silver. Although you shouldn't expect that in this competition anymore (more on that below), your placement could still rise or fall quite a bit.
We're halfway through, so about half of our scores cannot be undone. There's probably a competitor out there with an amazing solution, worthy of the top 10 but currently around position 100. In a different timeline, their solution could have won a gold medal or even some money. But their bad luck in this timeline is already baked into their score, making it unlikely they'd move to the top.
\nThere may also be competitors in the top 10 whose solutions are 50th place material. Their good luck is baked in, however, so they'll probably end up doing well. (Someone will probably hate reading this and downvote me for saying the obvious. That's humanity for you.)
\nBecause that was always how this competition would play out, it was a mistake to follow a strategy that depended on precision and .001 improvements. Hacking the objective function is that kind of thing. It could help you overfit the stage 1 leaderboard, but nothing more in my opinion.
\nIt was also a mistake to spend a lot of time tuning at the expense of engineering features or making an efficient submission. For this competition, I found that it was important to have a large variety of features and to fit a model to the entire dataset. CV can tell you what's most important, or where easy progress can be made.
", "rawMarkdown": "Have you tested your hypothesis with cross validation (CV)? It sounds like you're able to calculate the competition target multiple ways, and you have a model, and you should also have CV setup for the competition. So you could put the three things together and show us what you find.\n\nHere's my hypothesis: the effect of any strangeness in the objective function will be insignificant compared to noise. \n\nMy model is doing alright (12th place) and I didn't account for any details of the objective function. I just weighted my samples and used L2 cost. My only accounting for missing data was to fill forward. \n\nMore on noise:\n\nLast update, 6 people in positions 10-20 fell to >400. In addition, a competitor came from position 110 to position 4. \n\nBy my estimation, the error in our scores is still about .02, which would imply a couple of things:\n\n1. The top 50 scores are not very distinguishable. You singled out Tom Forbes, but his model is only doing well now because it did extremely well on the second update. For the third update, it placed around 8th by my estimate, and on the first update, it placed 99th (actual placement). The very high score on the second update was probably mostly luck.\n\n2. Your current placement is not very distinguishable from 30th place, which would be silver. Although you shouldn't expect that in this competition anymore (more on that below), your placement could still rise or fall quite a bit.\n\nWe're halfway through, so about half of our scores cannot be undone. There's probably a competitor out there with an amazing solution, worthy of the top 10 but currently around position 100. In a different timeline, their solution could have won a gold medal or even some money. But their bad luck in this timeline is already baked into their score, making it unlikely they'd move to the top.\n\nThere may also be competitors in the top 10 whose solutions are 50th place material. Their good luck is baked in, however, so they'll probably end up doing well. (Someone will probably hate reading this and downvote me for saying the obvious. That's humanity for you.)\n\nBecause that was always how this competition would play out, it was a mistake to follow a strategy that depended on precision and .001 improvements. Hacking the objective function is that kind of thing. It could help you overfit the stage 1 leaderboard, but nothing more in my opinion.\n\nIt was also a mistake to spend a lot of time tuning at the expense of engineering features or making an efficient submission. For this competition, I found that it was important to have a large variety of features and to fit a model to the entire dataset. CV can tell you what's most important, or where easy progress can be made.", "votes": 4, "replies": [ { "id": 1731295, "postDate": "2022-03-22T07:57:50.927Z", "content": "@nathanrm after so many years around here I have heard of CV before ;-)
\nI have singled out @tomforbes because 1) he has published a well aimed solution and 2) his solution is one of the only 6 of the leaderboard scores that are beating naive solution (train) score what to me implies that at least part of its predictive power has survived to the glitch.
\nAbout explanations why some of the rest of 2k solutions do better that others all bets are off. As you probably noticed data quality was not stable in train with some periods quite different to others. Whoever had their model weight those poor quality periods will have optimized good enough to do well in private leaderboard.
\nAbout possible future shakeups, of course, with current noise dynamics anyone could do anything with a single update. But on paper current #1 solution does have an edge.
\nApparently you don't find surprising that only 6 scores out of 2k in the leaderboard beat the naive solution training score. That makes further discussing pointless as I do think it is something objectively surprising, an anomaly, that adds to many other anomalies seen like strong negative correlations.
\nAlso apparently you dont' like that I single out Tom Forbes and not your solution. I hope I have made clear
\nwhy I am not impressed by solutions not beating naive score. But don't get me wrong, #12 is a great result also, your solution might in the end beat the naive score or even place higher and I wish you luck so, please, let me move on with this :-)
I gave up debating two weeks ago😂 And I will never join any kind of financial competitions on Kaggle anymore. Too much shakeup and the LB depends on too many things besides data modelling skills.
", "rawMarkdown": "I gave up debating two weeks ago😂 And I will never join any kind of financial competitions on Kaggle anymore. Too much shakeup and the LB depends on too many things besides data modelling skills.", "votes": 4, "replies": [ { "id": 1729713, "postDate": "2022-03-20T12:18:49.687Z", "content": "Well data modelling is usually a good skill to train (arguably more important than model training). If you are not interested there are some financial competition with anonymous feature (such as the on-going ubiquant). But not knowing the feature make it arguably worse: the concern you have here about target apply to the whole dataset (see for exemple: https://www.kaggle.com/code/lucasmorin/weird-patterns-in-unique-values-across-time-ids).
", "rawMarkdown": "Well data modelling is usually a good skill to train (arguably more important than model training). If you are not interested there are some financial competition with anonymous feature (such as the on-going ubiquant). But not knowing the feature make it arguably worse: the concern you have here about target apply to the whole dataset (see for exemple: https://www.kaggle.com/code/lucasmorin/weird-patterns-in-unique-values-across-time-ids). ", "votes": 2 }, { "id": 1730167, "postDate": "2022-03-21T01:42:22.667Z", "content": "I don't think we can do any engineering with anonymous features… I don't have too much expertise in finance. Since I am from NLP, I would rather join competitions like FeedbackPrize which ended a few days ago, and the ongoing NBME. The fact that I don't join financial competitions on Kaggle doesn't mean that I don't want to make money in finance. But with my current practice in stock market, I have known that a good prediction model isn't all. A good strategy or portfolio is the most important. And I will keep exploring things like that. After all it's a life time work😂.
", "rawMarkdown": "I don't think we can do any engineering with anonymous features... I don't have too much expertise in finance. Since I am from NLP, I would rather join competitions like FeedbackPrize which ended a few days ago, and the ongoing NBME. The fact that I don't join financial competitions on Kaggle doesn't mean that I don't want to make money in finance. But with my current practice in stock market, I have known that a good prediction model isn't all. A good strategy or portfolio is the most important. And I will keep exploring things like that. After all it's a life time work😂.", "votes": 1 } ] }, { "id": 1734809, "postDate": "2022-03-25T16:18:16.880Z", "content": "\n\nSo many comments/questions ignored.
\n
I feel the same way :(. There was very little input from the host during the competition and I found it frustrating as well as a departure from the way competitions have been conducted over the last decade (yikes!).
", "rawMarkdown": "> So many comments/questions ignored.\n\nI feel the same way :(. There was very little input from the host during the competition and I found it frustrating as well as a departure from the way competitions have been conducted over the last decade (yikes!). ", "votes": 1 }, { "id": 1731662, "postDate": "2022-03-22T15:27:54.243Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1731075, "author_name": "Nathaniel Maddux", "author_url": "", "post_date": "2022-03-22T01:25:35.327000", "content": "Have you tested your hypothesis with cross validation (CV)? It sounds like you're able to calculate the competition target multiple ways, and you have a model, and you should also have CV setup for the competition. So you could put the three things together and show us what you find.
\nHere's my hypothesis: the effect of any strangeness in the objective function will be insignificant compared to noise.
\nMy model is doing alright (12th place) and I didn't account for any details of the objective function. I just weighted my samples and used L2 cost. My only accounting for missing data was to fill forward.
\nMore on noise:
\nLast update, 6 people in positions 10-20 fell to >400. In addition, a competitor came from position 110 to position 4.
\nBy my estimation, the error in our scores is still about .02, which would imply a couple of things:
\nThe top 50 scores are not very distinguishable. You singled out Tom Forbes, but his model is only doing well now because it did extremely well on the second update. For the third update, it placed around 8th by my estimate, and on the first update, it placed 99th (actual placement). The very high score on the second update was probably mostly luck.
Your current placement is not very distinguishable from 30th place, which would be silver. Although you shouldn't expect that in this competition anymore (more on that below), your placement could still rise or fall quite a bit.
We're halfway through, so about half of our scores cannot be undone. There's probably a competitor out there with an amazing solution, worthy of the top 10 but currently around position 100. In a different timeline, their solution could have won a gold medal or even some money. But their bad luck in this timeline is already baked into their score, making it unlikely they'd move to the top.
\nThere may also be competitors in the top 10 whose solutions are 50th place material. Their good luck is baked in, however, so they'll probably end up doing well. (Someone will probably hate reading this and downvote me for saying the obvious. That's humanity for you.)
\nBecause that was always how this competition would play out, it was a mistake to follow a strategy that depended on precision and .001 improvements. Hacking the objective function is that kind of thing. It could help you overfit the stage 1 leaderboard, but nothing more in my opinion.
\nIt was also a mistake to spend a lot of time tuning at the expense of engineering features or making an efficient submission. For this competition, I found that it was important to have a large variety of features and to fit a model to the entire dataset. CV can tell you what's most important, or where easy progress can be made.
", "votes": 4, "replies": [ { "id": 1731295, "author_name": "miguel perez", "author_url": "", "post_date": "2022-03-22T07:57:50.927000", "content": "@nathanrm after so many years around here I have heard of CV before ;-)
\nI have singled out @tomforbes because 1) he has published a well aimed solution and 2) his solution is one of the only 6 of the leaderboard scores that are beating naive solution (train) score what to me implies that at least part of its predictive power has survived to the glitch.
\nAbout explanations why some of the rest of 2k solutions do better that others all bets are off. As you probably noticed data quality was not stable in train with some periods quite different to others. Whoever had their model weight those poor quality periods will have optimized good enough to do well in private leaderboard.
\nAbout possible future shakeups, of course, with current noise dynamics anyone could do anything with a single update. But on paper current #1 solution does have an edge.
\nApparently you don't find surprising that only 6 scores out of 2k in the leaderboard beat the naive solution training score. That makes further discussing pointless as I do think it is something objectively surprising, an anomaly, that adds to many other anomalies seen like strong negative correlations.
\nAlso apparently you dont' like that I single out Tom Forbes and not your solution. I hope I have made clear
\nwhy I am not impressed by solutions not beating naive score. But don't get me wrong, #12 is a great result also, your solution might in the end beat the naive score or even place higher and I wish you luck so, please, let me move on with this :-)
I gave up debating two weeks ago😂 And I will never join any kind of financial competitions on Kaggle anymore. Too much shakeup and the LB depends on too many things besides data modelling skills.
", "votes": 4, "replies": [ { "id": 1729713, "author_name": "Lucas Morin", "author_url": "", "post_date": "2022-03-20T12:18:49.687000", "content": "Well data modelling is usually a good skill to train (arguably more important than model training). If you are not interested there are some financial competition with anonymous feature (such as the on-going ubiquant). But not knowing the feature make it arguably worse: the concern you have here about target apply to the whole dataset (see for exemple: https://www.kaggle.com/code/lucasmorin/weird-patterns-in-unique-values-across-time-ids).
", "votes": 2, "replies": [] }, { "id": 1730167, "author_name": "ACMHIT", "author_url": "", "post_date": "2022-03-21T01:42:22.667000", "content": "I don't think we can do any engineering with anonymous features… I don't have too much expertise in finance. Since I am from NLP, I would rather join competitions like FeedbackPrize which ended a few days ago, and the ongoing NBME. The fact that I don't join financial competitions on Kaggle doesn't mean that I don't want to make money in finance. But with my current practice in stock market, I have known that a good prediction model isn't all. A good strategy or portfolio is the most important. And I will keep exploring things like that. After all it's a life time work😂.
", "votes": 1, "replies": [] } ] }, { "id": 1734809, "author_name": "Chase", "author_url": "", "post_date": "2022-03-25T16:18:16.880000", "content": "\n\nSo many comments/questions ignored.
\n
I feel the same way :(. There was very little input from the host during the competition and I found it frustrating as well as a departure from the way competitions have been conducted over the last decade (yikes!).
", "votes": 1, "replies": [] }, { "id": 1731662, "author_name": "", "author_url": "", "post_date": "2022-03-22T15:27:54.243000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1729197": "Yes, still one and a half month left till final update. But as it is quite clear that organizers don't plan to \ndedicate a minute to the issues raised I probably shouldn't waste much more time either.\n\nBefore moving on, I thought I would make a quick summary of what I think it has happened in this competition, for what is worth. \n\nIt is debatable to what extent what has happened can be considered part of the task of making our models generalize to unseen events/future periods. As @tomforbes has shown it was possible to prepare models for extreme dataset shift and if it was doable then it can be argued that it also can be expected to be done.\n\nAs a recap, label of this competition was generated with different weird decisions, one of them critically linked it to the amount of missing values in the data feed. The effect of this was so huge that if data quality significativaly differed of train dataset models were rendered useless. Unless data quality was accounted for at modeling time.\n\nIf this is solely what is happening we can not know without data but judging for the private LB results many competitors didn't expect data quality to significatively differ from what was seen during stage 1 **as it would have nothing to do with market forces but solely with the host data feed and both host and Kaggle's preprocessing pipeline**. At inference time, data quality did differ and many scores are weirdly and unexpectedly low/negative.\n\nSo, that regarding the comp. **The main point of debate here is to what extent data quality is a legit modeling objective**. Who knows, maybe comp host cleverly planned all this as a meta test. But it might be just good old mistakes and lack of attention to label design and data preparation. \n\nAs I say, I consider this debatable. As for myself, I wish I had been clever enough to see it coming and hedge it in the way current #1 solution has done. But at the same time in my opinion a competition where generalization is engineered not to happen is a competition of very limited interest for anyone, including the host btw, does the host even care about what Kagglers can do in this domain? \n\nLastly, another thing that has happened in this competition is that Kaggle staff has completely ignored legitimate concerns during and after the competition. So many comments/questions ignored. It would have been very easy to say \"data quality is part of the label and data quality of stage2 is poor, but we say that is legit\" or, instead \"we are going to double check data quality\". Or any educated easy answer that showed a minimal concern about doing things right and a minimal respect towards us Kagglers. In this regard I think it's easily the worst organized Kaggle competition I've seen up to date.\n\nAnyway, fun as it is to do a forensic data analysis without data it's probably better to stop spending energy on this. \n\nI thought this would make a short summary to wrap up and maybe help some other competitors still wondering about all this but yes, time to move on.", "1731075": "Have you tested your hypothesis with cross validation (CV)? It sounds like you're able to calculate the competition target multiple ways, and you have a model, and you should also have CV setup for the competition. So you could put the three things together and show us what you find.\n\nHere's my hypothesis: the effect of any strangeness in the objective function will be insignificant compared to noise. \n\nMy model is doing alright (12th place) and I didn't account for any details of the objective function. I just weighted my samples and used L2 cost. My only accounting for missing data was to fill forward. \n\nMore on noise:\n\nLast update, 6 people in positions 10-20 fell to >400. In addition, a competitor came from position 110 to position 4. \n\nBy my estimation, the error in our scores is still about .02, which would imply a couple of things:\n\n1. The top 50 scores are not very distinguishable. You singled out Tom Forbes, but his model is only doing well now because it did extremely well on the second update. For the third update, it placed around 8th by my estimate, and on the first update, it placed 99th (actual placement). The very high score on the second update was probably mostly luck.\n\n2. Your current placement is not very distinguishable from 30th place, which would be silver. Although you shouldn't expect that in this competition anymore (more on that below), your placement could still rise or fall quite a bit.\n\nWe're halfway through, so about half of our scores cannot be undone. There's probably a competitor out there with an amazing solution, worthy of the top 10 but currently around position 100. In a different timeline, their solution could have won a gold medal or even some money. But their bad luck in this timeline is already baked into their score, making it unlikely they'd move to the top.\n\nThere may also be competitors in the top 10 whose solutions are 50th place material. Their good luck is baked in, however, so they'll probably end up doing well. (Someone will probably hate reading this and downvote me for saying the obvious. That's humanity for you.)\n\nBecause that was always how this competition would play out, it was a mistake to follow a strategy that depended on precision and .001 improvements. Hacking the objective function is that kind of thing. It could help you overfit the stage 1 leaderboard, but nothing more in my opinion.\n\nIt was also a mistake to spend a lot of time tuning at the expense of engineering features or making an efficient submission. For this competition, I found that it was important to have a large variety of features and to fit a model to the entire dataset. CV can tell you what's most important, or where easy progress can be made.", "1729380": "I gave up debating two weeks ago😂 And I will never join any kind of financial competitions on Kaggle anymore. Too much shakeup and the LB depends on too many things besides data modelling skills.", "1734809": "> So many comments/questions ignored.\n\nI feel the same way :(. There was very little input from the host during the competition and I found it frustrating as well as a departure from the way competitions have been conducted over the last decade (yikes!). ", "1731662": "" } }