{
  "id": 5826,
  "title": "logistic regression, strange behavior",
  "url": "/competitions/digit-recognizer/discussion/5826",
  "author_name": "ivarg",
  "post_date": "2013-09-20T11:08:03.957000",
  "votes": 0,
  "comment_count": 0,
  "views": null,
  "content": "<p>Hello,</p>\n<p>I have been trying to write my own logistic regression based on <a href=\"http://deeplearning.net/software/theano/\">theano</a>, and their <a href=\"http://www.deeplearning.net/tutorial/logreg.html#logreg\">tutorial example</a>. I do mini-batch SGD, and have experimented with different batch sizes, where a batch size of 200 gives an initial expected reduction in cost during the first epoch, i.e. when traversing the training set (40000) for the first time.</p>\n<p>However, during the second, third and fourth epochs, the cost updates become very noisy and express an identical pattern for each epoch. I can't figure out why.</p>\n<p>I have uploaded plots from all four epochs</p>\n<p>EDIT</p>\n<p>Actually, when using same y axis, it seems that the same pattern repeats in all four plots, which confuses me even more...</p>",
  "messages": [
    {
      "id": 31244,
      "postDate": "2013-09-20T11:08:03.957Z",
      "content": "<p>Hello,</p>\n<p>I have been trying to write my own logistic regression based on <a href=\"http://deeplearning.net/software/theano/\">theano</a>, and their <a href=\"http://www.deeplearning.net/tutorial/logreg.html#logreg\">tutorial example</a>. I do mini-batch SGD, and have experimented with different batch sizes, where a batch size of 200 gives an initial expected reduction in cost during the first epoch, i.e. when traversing the training set (40000) for the first time.</p>\n<p>However, during the second, third and fourth epochs, the cost updates become very noisy and express an identical pattern for each epoch. I can't figure out why.</p>\n<p>I have uploaded plots from all four epochs</p>\n<p>EDIT</p>\n<p>Actually, when using same y axis, it seems that the same pattern repeats in all four plots, which confuses me even more...</p>"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "31244": ""
  }
}