{
  "id": 327740,
  "title": "Fractal markdown distribution",
  "url": "/competitions/AI4Code/discussion/327740",
  "author_name": "jagofc",
  "post_date": "2022-05-28T20:26:21.583000",
  "votes": 21,
  "comment_count": 4,
  "views": 0,
  "content": "<p>So you probably won't find this useful, but it <em>is</em> kind of cool…</p>\n<p>The empirical markdown cell proportions (# markdown cells / # cells) for the notebooks have this funky fractal-esque distribution:</p>\n<p><img src=\"https://i.postimg.cc/KvNwt9Qm/md-proportions.png\" alt=\"markdown proportions\"></p>",
  "messages": [
    {
      "id": 1804370,
      "postDate": "2022-05-28T20:26:21.583Z",
      "content": "<p>So you probably won't find this useful, but it <em>is</em> kind of cool…</p>\n<p>The empirical markdown cell proportions (# markdown cells / # cells) for the notebooks have this funky fractal-esque distribution:</p>\n<p><img src=\"https://i.postimg.cc/KvNwt9Qm/md-proportions.png\" alt=\"markdown proportions\"></p>",
      "rawMarkdown": "So you probably won't find this useful, but it *is* kind of cool...\n\nThe empirical markdown cell proportions (# markdown cells / # cells) for the notebooks have this funky fractal-esque distribution:\n\n![markdown proportions](https://i.postimg.cc/KvNwt9Qm/md-proportions.png)",
      "votes": 20
    },
    {
      "id": 1835824,
      "postDate": "2022-06-28T05:52:26.943Z",
      "content": "<p>Very interesting! What are those long lines at various intervals ? </p>",
      "rawMarkdown": "Very interesting! What are those long lines at various intervals ? ",
      "replies": [
        {
          "id": 1838514,
          "postDate": "2022-06-30T15:43:04.483Z",
          "rawMarkdown": "",
          "votes": 1,
          "isDeleted": true
        },
        {
          "id": 1838830,
          "postDate": "2022-06-30T21:40:29.073Z",
          "content": "<p>Hey! Apologies for the slow response, busy week! A quick comment to compliment <a href=\"https://www.kaggle.com/patrickdwyerhoops\" target=\"_blank\">@patrickdwyerhoops</a>… </p>\n<p>There are three factors giving rise to this distribution:</p>\n<ol>\n<li>For a notebook of length N there are only a finite set of possible (# md cells / # cells) (i.e. 1/N, 2/N, …, N/N).</li>\n<li>The distribution of notebook lengths N has most of its mass on relatively small positive integers.</li>\n<li>There are typically fewer markdown cells than code cells.</li>\n</ol>\n<p>These three factors combine to highlight proportions with a small denominator, peaked at 1/2 because there are many ways of making a proportion of 1/2(= 2/4 = 3/6 = 4/8), but also with strong peaks at 1/3, 1/4, for similar reasons. <br>\nThe downward trend is roughly linear because (heuristically) there are about half as many ways for a notebook to have proportion 1/4 as there are to have a proportion 1/2 so the peak at 1/4 is half the height of the peak at 1/2 - those peaks lie on the line y=Mx for a fixed M~10000.<br>\nThis also explains the separation phenomenon between peaks - there are relatively few notebooks that are long enough to permit a denominator giving a (# md cells / # cells) proportion very near to 1/2, as highlighted by <a href=\"https://www.kaggle.com/patrickdwyerhoops\" target=\"_blank\">@patrickdwyerhoops</a>.</p>\n<p>So it's a mixture of the properties of the rational numbers, the finiteness of notebook lengths and the tendencies of coders to do less storytelling than coding ;)</p>",
          "rawMarkdown": "Hey! Apologies for the slow response, busy week! A quick comment to compliment @patrickdwyerhoops... \n\nThere are three factors giving rise to this distribution:\n\n1. For a notebook of length N there are only a finite set of possible (# md cells / # cells) (i.e. 1/N, 2/N, …, N/N).\n2. The distribution of notebook lengths N has most of its mass on relatively small positive integers.\n3. There are typically fewer markdown cells than code cells.\n\nThese three factors combine to highlight proportions with a small denominator, peaked at 1/2 because there are many ways of making a proportion of 1/2(= 2/4 = 3/6 = 4/8), but also with strong peaks at 1/3, 1/4, for similar reasons. \nThe downward trend is roughly linear because (heuristically) there are about half as many ways for a notebook to have proportion 1/4 as there are to have a proportion 1/2 so the peak at 1/4 is half the height of the peak at 1/2 - those peaks lie on the line y=Mx for a fixed M~10000.\nThis also explains the separation phenomenon between peaks - there are relatively few notebooks that are long enough to permit a denominator giving a (# md cells / # cells) proportion very near to 1/2, as highlighted by @patrickdwyerhoops.\n\nSo it's a mixture of the properties of the rational numbers, the finiteness of notebook lengths and the tendencies of coders to do less storytelling than coding ;)",
          "votes": 1
        }
      ]
    },
    {
      "id": 1831207,
      "postDate": "2022-06-24T03:28:15.783Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1835824,
      "author_name": "theDoors",
      "author_url": "",
      "post_date": "2022-06-28T05:52:26.943000",
      "content": "<p>Very interesting! What are those long lines at various intervals ? </p>",
      "votes": 0,
      "replies": [
        {
          "id": 1838514,
          "author_name": "",
          "author_url": "",
          "post_date": "2022-06-30T15:43:04.483000",
          "content": "",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1838830,
          "author_name": "jagofc",
          "author_url": "",
          "post_date": "2022-06-30T21:40:29.073000",
          "content": "<p>Hey! Apologies for the slow response, busy week! A quick comment to compliment <a href=\"https://www.kaggle.com/patrickdwyerhoops\" target=\"_blank\">@patrickdwyerhoops</a>… </p>\n<p>There are three factors giving rise to this distribution:</p>\n<ol>\n<li>For a notebook of length N there are only a finite set of possible (# md cells / # cells) (i.e. 1/N, 2/N, …, N/N).</li>\n<li>The distribution of notebook lengths N has most of its mass on relatively small positive integers.</li>\n<li>There are typically fewer markdown cells than code cells.</li>\n</ol>\n<p>These three factors combine to highlight proportions with a small denominator, peaked at 1/2 because there are many ways of making a proportion of 1/2(= 2/4 = 3/6 = 4/8), but also with strong peaks at 1/3, 1/4, for similar reasons. <br>\nThe downward trend is roughly linear because (heuristically) there are about half as many ways for a notebook to have proportion 1/4 as there are to have a proportion 1/2 so the peak at 1/4 is half the height of the peak at 1/2 - those peaks lie on the line y=Mx for a fixed M~10000.<br>\nThis also explains the separation phenomenon between peaks - there are relatively few notebooks that are long enough to permit a denominator giving a (# md cells / # cells) proportion very near to 1/2, as highlighted by <a href=\"https://www.kaggle.com/patrickdwyerhoops\" target=\"_blank\">@patrickdwyerhoops</a>.</p>\n<p>So it's a mixture of the properties of the rational numbers, the finiteness of notebook lengths and the tendencies of coders to do less storytelling than coding ;)</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1831207,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-06-24T03:28:15.783000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1804370": "So you probably won't find this useful, but it *is* kind of cool...\n\nThe empirical markdown cell proportions (# markdown cells / # cells) for the notebooks have this funky fractal-esque distribution:\n\n![markdown proportions](https://i.postimg.cc/KvNwt9Qm/md-proportions.png)",
    "1835824": "Very interesting! What are those long lines at various intervals ? ",
    "1831207": ""
  }
}