{
  "id": 111601,
  "title": "accuracy on validation set varies at each run using keras cnn",
  "url": "/competitions/Kannada-MNIST/discussion/111601",
  "author_name": "",
  "post_date": "2019-10-07T09:31:06.402293100Z",
  "votes": 2,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I never user keras in R before this competition so I was surprised to notice that different run on identical data produce different results despite I set set.seed() just on top of the training model.</p>\n\n<p>I was using GPU so I may guess the reason why is that but I'd like to ask fellow kagglelers if anybody has a hint on this issue.</p>",
  "messages": [
    {
      "id": "643224",
      "postDate": "10/07/2019 09:31:06",
      "content": "<p>I never user keras in R before this competition so I was surprised to notice that different run on identical data produce different results despite I set set.seed() just on top of the training model.</p>\n\n<p>I was using GPU so I may guess the reason why is that but I'd like to ask fellow kagglelers if anybody has a hint on this issue.</p>",
      "rawMarkdown": "I never user keras in R before this competition so I was surprised to notice that different run on identical data produce different results despite I set set.seed() just on top of the training model.\n\nI was using GPU so I may guess the reason why is that but I'd like to ask fellow kagglelers if anybody has a hint on this issue.",
      "votes": null
    },
    {
      "id": "643572",
      "postDate": "10/07/2019 16:26:22",
      "content": "<p>yep. I learned that lesson already. using learn.one_cycle() from fast.ai\nset.seed() it the same - depending on the number you use as input it will generate random numbers - but actually are the same number for each seed value - which in fact are not random.\nBut anyway based on that seed it's taking a specific set (from those 0.8) to learn. So depending on that seed value - the selection 80% will be slightly different. I am very skeptical with every provided data source - so there might be some samples designated to confuse the learning model.\nnot sure I helped much, I'll keep an eye on your post. Voted up.\nregards,\nA</p>",
      "rawMarkdown": "yep. I learned that lesson already. using learn.one_cycle() from fast.ai\nset.seed() it the same - depending on the number you use as input it will generate random numbers - but actually are the same number for each seed value - which in fact are not random.\nBut anyway based on that seed it's taking a specific set (from those 0.8) to learn. So depending on that seed value - the selection 80% will be slightly different. I am very skeptical with every provided data source - so there might be some samples designated to confuse the learning model.\nnot sure I helped much, I'll keep an eye on your post. Voted up.\nregards,\nA",
      "votes": null
    },
    {
      "id": "644242",
      "postDate": "10/08/2019 14:25:11",
      "content": "<p>watching this closed too. </p>",
      "rawMarkdown": "watching this closed too.",
      "votes": null
    },
    {
      "id": "651850",
      "postDate": "10/18/2019 02:34:16",
      "content": "<p>It's tensorflow features/bugs(it's hard to say what is it really). You can obtain the same results in two independent running only if would not be used parallelism (used 1 core of cpu only), but and this not guaranteed the same results. I can advise you use pytorch if you want have the same results.\nP.s. This video can help you understand why it's happening.</p>",
      "rawMarkdown": "It's tensorflow features/bugs(it's hard to say what is it really). You can obtain the same results in two independent running only if would not be used parallelism (used 1 core of cpu only), but and this not guaranteed the same results. I can advise you use pytorch if you want have the same results.\nP.s. This video can help you understand why it's happening.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 643572,
      "author_name": "zinovadr",
      "author_url": "",
      "post_date": "10/07/2019 16:26:22",
      "content": "<p>yep. I learned that lesson already. using learn.one_cycle() from fast.ai\nset.seed() it the same - depending on the number you use as input it will generate random numbers - but actually are the same number for each seed value - which in fact are not random.\nBut anyway based on that seed it's taking a specific set (from those 0.8) to learn. So depending on that seed value - the selection 80% will be slightly different. I am very skeptical with every provided data source - so there might be some samples designated to confuse the learning model.\nnot sure I helped much, I'll keep an eye on your post. Voted up.\nregards,\nA</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 644242,
      "author_name": "zinovadr",
      "author_url": "",
      "post_date": "10/08/2019 14:25:11",
      "content": "<p>watching this closed too. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 651850,
      "author_name": "miklgr500",
      "author_url": "",
      "post_date": "10/18/2019 02:34:16",
      "content": "<p>It's tensorflow features/bugs(it's hard to say what is it really). You can obtain the same results in two independent running only if would not be used parallelism (used 1 core of cpu only), but and this not guaranteed the same results. I can advise you use pytorch if you want have the same results.\nP.s. This video can help you understand why it's happening.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "643224": "I never user keras in R before this competition so I was surprised to notice that different run on identical data produce different results despite I set set.seed() just on top of the training model.\n\nI was using GPU so I may guess the reason why is that but I'd like to ask fellow kagglelers if anybody has a hint on this issue.",
    "643572": "yep. I learned that lesson already. using learn.one_cycle() from fast.ai\nset.seed() it the same - depending on the number you use as input it will generate random numbers - but actually are the same number for each seed value - which in fact are not random.\nBut anyway based on that seed it's taking a specific set (from those 0.8) to learn. So depending on that seed value - the selection 80% will be slightly different. I am very skeptical with every provided data source - so there might be some samples designated to confuse the learning model.\nnot sure I helped much, I'll keep an eye on your post. Voted up.\nregards,\nA",
    "644242": "watching this closed too.",
    "651850": "It's tensorflow features/bugs(it's hard to say what is it really). You can obtain the same results in two independent running only if would not be used parallelism (used 1 core of cpu only), but and this not guaranteed the same results. I can advise you use pytorch if you want have the same results.\nP.s. This video can help you understand why it's happening."
  },
  "source": "meta"
}