{
  "id": 17020,
  "title": "My submission (score 0.00512)",
  "content": "<p>I will release my scripts in a few steps, as I have to tidy up the code, and write up a description of what I've been doing. My model consisted of 4 stages:</p>\n\n<p>Stage 1: background removal using information leakage\nStage 2: several different models using DNN, CNN, and boosted trees\nStage 3: ensembling the models via xgboost\nStage 4: using yet another information leakage on the text to halve the score</p>\n\n<p>You can read about Stage 1, my background removal script, at <a href=\"http://colinpriest.com/2015/10/15/denoising-dirty-documents-part-9/\">http://colinpriest.com/2015/10/15/denoising-dirty-documents-part-9/</a> , which is a much generic and comprehensive approach than that presented in Kaggle scripts. Of particular interest is the adjustment for contrast constraints that fixes the problem of faded text where the stains are dark.</p>\n\n<p>My next post will start to cover the Stage 2 models. I found that the deep neural networks (DNN) performed the best, even without adding many layers. The convolutional neural network (CNN) took quite a while to get working, but didn't perform as well as the DNN, probably because it lacked any fully connected layers.</p>\n\n<p>Colin</p>",
  "comments": [
    {
      "id": 98851,
      "author_name": "Colin",
      "author_url": "",
      "post_date": "2015-11-15T04:03:46.230000",
      "content": "<p>Hi All,</p>\n\n<p>I've published my final two blogs about this competition. They explain the deep learning models that I built (which were the most successful models that I tried), and the second information leak, which relates to repeated text (and which managed to halve my leaderboard score!).</p>\n\n<p>You can read them at:</p>\n\n<p><a href=\"http://colinpriest.com/2015/11/08/denoising-dirty-documents-part-11/\">http://colinpriest.com/2015/11/08/denoising-dirty-documents-part-11/</a></p>\n\n<p><a href=\"http://colinpriest.com/2015/11/15/an-even-dozen-denoising-dirty-documents-part-12/\">http://colinpriest.com/2015/11/15/an-even-dozen-denoising-dirty-documents-part-12/</a> </p>\n\n<p>Colin</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 97904,
      "author_name": "Colin",
      "author_url": "",
      "post_date": "2015-11-01T10:21:23.840000",
      "content": "<p>My latest blog post about this competition has been published. You can read about my convolutional neural network model at <a href=\"http://colinpriest.com/2015/11/01/denoising-dirty-documents-part-10/\">http://colinpriest.com/2015/11/01/denoising-dirty-documents-part-10/</a> . I used a convolutional neural network with 3 hidden layers, each layer having 25 filters.  While this model performed well, I feel that it could have performed better if I had used a different structure that included one or more fully connected layers before the output layer.</p>\n\n<p>Unlike the rest of my competition submission, this model was coded in Python so that I could use the Theano library. I used Daniel Nouri's nolearn library to simplify the coding. This was my first time ever coding in Python and also my first time ever using convolutional neural networks, so the learning curve was steep and it took me a while to get it working. The full Python script has been included.</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 100305,
      "author_name": "Ana Lina Morelia",
      "author_url": "",
      "post_date": "2015-12-04T23:57:40.447000",
      "content": "<p>Don't miss the blog post summarizing Colin's excellent tutorial series (with links to code and the full tutorials) on no free hunch. Take a look <a href=\"http://blog.kaggle.com/2015/12/04/image-processing-machine-learning-in-r-denoising-dirty-documents-tutorial-series/\">here</a>. The competition may be over, but the opportunities to learn haven't finished!</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "comment_count": 3,
  "messages": [
    {
      "id": 96257,
      "postDate": "2015-10-15T10:37:39.527Z",
      "content": "<p>I will release my scripts in a few steps, as I have to tidy up the code, and write up a description of what I've been doing. My model consisted of 4 stages:</p>\n\n<p>Stage 1: background removal using information leakage\nStage 2: several different models using DNN, CNN, and boosted trees\nStage 3: ensembling the models via xgboost\nStage 4: using yet another information leakage on the text to halve the score</p>\n\n<p>You can read about Stage 1, my background removal script, at <a href=\"http://colinpriest.com/2015/10/15/denoising-dirty-documents-part-9/\">http://colinpriest.com/2015/10/15/denoising-dirty-documents-part-9/</a> , which is a much generic and comprehensive approach than that presented in Kaggle scripts. Of particular interest is the adjustment for contrast constraints that fixes the problem of faded text where the stains are dark.</p>\n\n<p>My next post will start to cover the Stage 2 models. I found that the deep neural networks (DNN) performed the best, even without adding many layers. The convolutional neural network (CNN) took quite a while to get working, but didn't perform as well as the DNN, probably because it lacked any fully connected layers.</p>\n\n<p>Colin</p>",
      "rawMarkdown": "I will release my scripts in a few steps, as I have to tidy up the code, and write up a description of what I've been doing. My model consisted of 4 stages:\r\n\r\nStage 1: background removal using information leakage\r\nStage 2: several different models using DNN, CNN, and boosted trees\r\nStage 3: ensembling the models via xgboost\r\nStage 4: using yet another information leakage on the text to halve the score\r\n\r\nYou can read about Stage 1, my background removal script, at http://colinpriest.com/2015/10/15/denoising-dirty-documents-part-9/ , which is a much generic and comprehensive approach than that presented in Kaggle scripts. Of particular interest is the adjustment for contrast constraints that fixes the problem of faded text where the stains are dark.\r\n\r\nMy next post will start to cover the Stage 2 models. I found that the deep neural networks (DNN) performed the best, even without adding many layers. The convolutional neural network (CNN) took quite a while to get working, but didn't perform as well as the DNN, probably because it lacked any fully connected layers.\r\n\r\nColin",
      "votes": 6
    },
    {
      "id": 98851,
      "postDate": "2015-11-15T04:03:46.230Z",
      "content": "<p>Hi All,</p>\n\n<p>I've published my final two blogs about this competition. They explain the deep learning models that I built (which were the most successful models that I tried), and the second information leak, which relates to repeated text (and which managed to halve my leaderboard score!).</p>\n\n<p>You can read them at:</p>\n\n<p><a href=\"http://colinpriest.com/2015/11/08/denoising-dirty-documents-part-11/\">http://colinpriest.com/2015/11/08/denoising-dirty-documents-part-11/</a></p>\n\n<p><a href=\"http://colinpriest.com/2015/11/15/an-even-dozen-denoising-dirty-documents-part-12/\">http://colinpriest.com/2015/11/15/an-even-dozen-denoising-dirty-documents-part-12/</a> </p>\n\n<p>Colin</p>",
      "rawMarkdown": "Hi All,\r\n\r\nI've published my final two blogs about this competition. They explain the deep learning models that I built (which were the most successful models that I tried), and the second information leak, which relates to repeated text (and which managed to halve my leaderboard score!).\r\n\r\nYou can read them at:\r\n\r\nhttp://colinpriest.com/2015/11/08/denoising-dirty-documents-part-11/\r\n\r\nhttp://colinpriest.com/2015/11/15/an-even-dozen-denoising-dirty-documents-part-12/ \r\n\r\nColin",
      "votes": 3
    },
    {
      "id": 97904,
      "postDate": "2015-11-01T10:21:23.840Z",
      "content": "<p>My latest blog post about this competition has been published. You can read about my convolutional neural network model at <a href=\"http://colinpriest.com/2015/11/01/denoising-dirty-documents-part-10/\">http://colinpriest.com/2015/11/01/denoising-dirty-documents-part-10/</a> . I used a convolutional neural network with 3 hidden layers, each layer having 25 filters.  While this model performed well, I feel that it could have performed better if I had used a different structure that included one or more fully connected layers before the output layer.</p>\n\n<p>Unlike the rest of my competition submission, this model was coded in Python so that I could use the Theano library. I used Daniel Nouri's nolearn library to simplify the coding. This was my first time ever coding in Python and also my first time ever using convolutional neural networks, so the learning curve was steep and it took me a while to get it working. The full Python script has been included.</p>",
      "rawMarkdown": "My latest blog post about this competition has been published. You can read about my convolutional neural network model at http://colinpriest.com/2015/11/01/denoising-dirty-documents-part-10/ . I used a convolutional neural network with 3 hidden layers, each layer having 25 filters.  While this model performed well, I feel that it could have performed better if I had used a different structure that included one or more fully connected layers before the output layer.\r\n\r\nUnlike the rest of my competition submission, this model was coded in Python so that I could use the Theano library. I used Daniel Nouri's nolearn library to simplify the coding. This was my first time ever coding in Python and also my first time ever using convolutional neural networks, so the learning curve was steep and it took me a while to get it working. The full Python script has been included.",
      "votes": 3
    },
    {
      "id": 100305,
      "postDate": "2015-12-04T23:57:40.447Z",
      "content": "<p>Don't miss the blog post summarizing Colin's excellent tutorial series (with links to code and the full tutorials) on no free hunch. Take a look <a href=\"http://blog.kaggle.com/2015/12/04/image-processing-machine-learning-in-r-denoising-dirty-documents-tutorial-series/\">here</a>. The competition may be over, but the opportunities to learn haven't finished!</p>",
      "rawMarkdown": "Don't miss the blog post summarizing Colin's excellent tutorial series (with links to code and the full tutorials) on no free hunch. Take a look [here][1]. The competition may be over, but the opportunities to learn haven't finished!\r\n\r\n\r\n  [1]: http://blog.kaggle.com/2015/12/04/image-processing-machine-learning-in-r-denoising-dirty-documents-tutorial-series/"
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T14:05:43Z",
  "competition_id": 4406,
  "competition_slug": "denoising-dirty-documents",
  "team_id": 198097,
  "solution_url": "/competitions/denoising-dirty-documents/writeups/colin-my-submission-score-0-00512"
}