{
  "id": 24018,
  "title": "Categories and the power of 2",
  "url": "/competitions/bosch-production-line-performance/discussion/24018",
  "author_name": "",
  "post_date": "2016-10-01T21:17:36.627Z",
  "votes": 4,
  "comment_count": 5,
  "views": 645,
  "content": "<p>So I was looking through my categorical data and it sure seemed like powers of two were popping up (a lot).</p>\n\n<p>Looking through my categorical data with multicolumns we have T2, T4, T32, T64, T128.  There seems to never be numbers in between them.  So i wasting my afternoon looking at this and thinking, those category labels cannot be arbitrary.  Some of them of silly long which seems ineffcient.  So what do they mean?</p>\n\n<p>I'm not sure just yet, but I found something very interesting.  It appears a large number of them are related to powers of 2.  I had calculated all the prime factors of the categories and it seemed like waaay to many were nice even powers of 2, very few were powers of 5 and many of them were very close to large powers of 2.   For example we have T33554432, T33554448, and T33554944.  Well the first one is 2^25 (yes i am a total math/physics nerd).   And the second one is just 16 more, but it factors kinda strange, but what the number is actualy (2^25+2^4)  the final one is (2^25+2^9)</p>\n\n<p>Now I first found this in the negative numbers--those struck me as really odd.  Who makes a category T-25151251?  Seems either inconsistent or it means something.  Well the most negative of those categories is -(2^31)=-2147483648.  Once again we can construct the near numbers through simple powers of 2. (-2147482944, -2147482816, -2147482688, -2147482432, -2147482176,-2147481664)</p>\n\n<p>As to WHAT it actually means--I am not sure--this borders on data dredging, but these categories seem to be more than just labels.  There appears to be some sort of numeric information in them.</p>\n\n<p>The following pure powers of 2 appear directly appear as categories: (2^N where N is):\n0,1,2,3,4,5,6,7,8,9,14,16,18,19,24,25,27,28,(-31).</p>\n\n<p>Someone owes me a beer or something =)</p>",
  "messages": [
    {
      "id": "137471",
      "postDate": "10/01/2016 21:17:36",
      "content": "<p>So I was looking through my categorical data and it sure seemed like powers of two were popping up (a lot).</p>\n\n<p>Looking through my categorical data with multicolumns we have T2, T4, T32, T64, T128.  There seems to never be numbers in between them.  So i wasting my afternoon looking at this and thinking, those category labels cannot be arbitrary.  Some of them of silly long which seems ineffcient.  So what do they mean?</p>\n\n<p>I'm not sure just yet, but I found something very interesting.  It appears a large number of them are related to powers of 2.  I had calculated all the prime factors of the categories and it seemed like waaay to many were nice even powers of 2, very few were powers of 5 and many of them were very close to large powers of 2.   For example we have T33554432, T33554448, and T33554944.  Well the first one is 2^25 (yes i am a total math/physics nerd).   And the second one is just 16 more, but it factors kinda strange, but what the number is actualy (2^25+2^4)  the final one is (2^25+2^9)</p>\n\n<p>Now I first found this in the negative numbers--those struck me as really odd.  Who makes a category T-25151251?  Seems either inconsistent or it means something.  Well the most negative of those categories is -(2^31)=-2147483648.  Once again we can construct the near numbers through simple powers of 2. (-2147482944, -2147482816, -2147482688, -2147482432, -2147482176,-2147481664)</p>\n\n<p>As to WHAT it actually means--I am not sure--this borders on data dredging, but these categories seem to be more than just labels.  There appears to be some sort of numeric information in them.</p>\n\n<p>The following pure powers of 2 appear directly appear as categories: (2^N where N is):\n0,1,2,3,4,5,6,7,8,9,14,16,18,19,24,25,27,28,(-31).</p>\n\n<p>Someone owes me a beer or something =)</p>",
      "rawMarkdown": "So I was looking through my categorical data and it sure seemed like powers of two were popping up (a lot).\r\n\r\nLooking through my categorical data with multicolumns we have T2, T4, T32, T64, T128.  There seems to never be numbers in between them.  So i wasting my afternoon looking at this and thinking, those category labels cannot be arbitrary.  Some of them of silly long which seems ineffcient.  So what do they mean?\r\n\r\nI'm not sure just yet, but I found something very interesting.  It appears a large number of them are related to powers of 2.  I had calculated all the prime factors of the categories and it seemed like waaay to many were nice even powers of 2, very few were powers of 5 and many of them were very close to large powers of 2.   For example we have T33554432, T33554448, and T33554944.  Well the first one is 2^25 (yes i am a total math/physics nerd).   And the second one is just 16 more, but it factors kinda strange, but what the number is actualy (2^25+2^4)  the final one is (2^25+2^9)\r\n\r\nNow I first found this in the negative numbers--those struck me as really odd.  Who makes a category T-25151251?  Seems either inconsistent or it means something.  Well the most negative of those categories is -(2^31)=-2147483648.  Once again we can construct the near numbers through simple powers of 2. (-2147482944, -2147482816, -2147482688, -2147482432, -2147482176,-2147481664)\r\n\r\nAs to WHAT it actually means--I am not sure--this borders on data dredging, but these categories seem to be more than just labels.  There appears to be some sort of numeric information in them.\r\n\r\nThe following pure powers of 2 appear directly appear as categories: (2^N where N is):\r\n0,1,2,3,4,5,6,7,8,9,14,16,18,19,24,25,27,28,(-31).\r\n\r\nSomeone owes me a beer or something =)",
      "votes": null
    },
    {
      "id": "137473",
      "postDate": "10/01/2016 21:24:33",
      "content": "<p>How I looked at this:</p>\n\n<p>from my previous published kernel:</p>\n\n<pre><code>import pandas as pd\nimport numpy as np\nimport collections\ndata=pd.read_csv(&quot;./kaggle/Bosch/data/train_categorical.csv&quot;,chunksize=100000, dtype=str ,usecols=list(range(1,2141)))\nuniques = collections.defaultdict(set)\n\n\nfor chunk in data: \n    for col in chunk:\n        uniques[col] = uniques[col].union(chunk[col][chunk[col].notnull()].unique())\n</code></pre>\n\n<p>Then I looked at columns with more than 1 category:</p>\n\n<pre><code>empty_cols=[]\nsingle_cols=[]\nmulti_cols=[]\nfor key in uniques:\n    if len(uniques[key])==0:\n        empty_cols.append(key)\n    elif len(uniques[key])==1:\n        single_cols.append(key)\n    else:\n        multi_cols.append(key)\n</code></pre>\n\n<p>Then I put them all together:\ncats=[]</p>\n\n<pre><code>for item in multi_cols:\n    for thing in uniques[item]:\n        cats.append(thing)\nset(cats)\n</code></pre>\n\n<p>if you want a count of each column (just for giggles cause you like to stare at numbers like me:</p>\n\n<pre><code>cat_cnt=dict()\nfor item in cats:\n    if item in cat_cnt:\n        cat_cnt[item] += 1\n    else:\n        cat_cnt[item] = 1\n</code></pre>",
      "rawMarkdown": "How I looked at this:\r\n\r\nfrom my previous published kernel:\r\n\r\n    import pandas as pd\r\n    import numpy as np\r\n    import collections\r\n    data=pd.read_csv(\"./kaggle/Bosch/data/train_categorical.csv\",chunksize=100000, dtype=str ,usecols=list(range(1,2141)))\r\n    uniques = collections.defaultdict(set)\r\n   \r\n  \r\n    for chunk in data: \r\n        for col in chunk:\r\n            uniques[col] = uniques[col].union(chunk[col][chunk[col].notnull()].unique())\r\n\r\nThen I looked at columns with more than 1 category:\r\n\r\n    empty_cols=[]\r\n    single_cols=[]\r\n    multi_cols=[]\r\n    for key in uniques:\r\n        if len(uniques[key])==0:\r\n            empty_cols.append(key)\r\n        elif len(uniques[key])==1:\r\n            single_cols.append(key)\r\n        else:\r\n            multi_cols.append(key)\r\nThen I put them all together:\r\ncats=[]\r\n\r\n    for item in multi_cols:\r\n        for thing in uniques[item]:\r\n            cats.append(thing)\r\n    set(cats)\r\n\r\nif you want a count of each column (just for giggles cause you like to stare at numbers like me:\r\n\r\n    cat_cnt=dict()\r\n    for item in cats:\r\n        if item in cat_cnt:\r\n            cat_cnt[item] += 1\r\n        else:\r\n            cat_cnt[item] = 1",
      "votes": null
    },
    {
      "id": "137476",
      "postDate": "10/01/2016 21:35:54",
      "content": "<p>Have you made any use of it ?:) Interesting findings tho</p>",
      "rawMarkdown": "Have you made any use of it ?:) Interesting findings tho",
      "votes": null
    },
    {
      "id": "137481",
      "postDate": "10/02/2016 00:57:04",
      "content": "<p>LOL, I noticed those 2^x values early on. I think it is an artifact of how they coded these variables (see below). There are only 93 individual values, and most columns use only one value. T1 is the most common value. I think the real question is whether T1 means the same thing for each column? Also, is there something linking columns that use value T4?</p>\n\n<p>32 bit binary representation. In this table, Value is the character variable value. Total is the count of times that value appears in the character data table. Cols is the number of columns where that variable appears at least once. Binary is a binary representation of the numerical value. I used the absolute value of negative numbers. </p>\n\n<pre><code>          Value  Total Cols                           Binary\n1            T1 245319 1319 00000000000000000000000000000001\n2            T2  25463  121 00000000000000000000000000000010\n3            T3   5461   30 00000000000000000000000000000011\n4            T4  14137  114 00000000000000000000000000000100\n5            T5   2945   16 00000000000000000000000000000101\n6            T6    964    4 00000000000000000000000000000110\n7            T7    624    4 00000000000000000000000000000111\n8            T8   9687   49 00000000000000000000000000001000\n9            T9    354    2 00000000000000000000000000001001\n10          T16  18768  110 00000000000000000000000000010000\n11          T24   3060   21 00000000000000000000000000011000\n12          T32  13641   91 00000000000000000000000000100000\n13          T48   6135   37 00000000000000000000000000110000\n14          T52   3406   13 00000000000000000000000000110100\n15          T56   3211   13 00000000000000000000000000111000\n16          T64   9307   51 00000000000000000000000001000000\n17          T96   5205   36 00000000000000000000000001100000\n18          T97    239    1 00000000000000000000000001100001\n19          T98    530    2 00000000000000000000000001100010\n20         T128   8280   36 00000000000000000000000010000000\n21         T143    418    3 00000000000000000000000010001111\n22         T145    437    3 00000000000000000000000010010001\n23         T256   1338    9 00000000000000000000000100000000\n24         T492    217    1 00000000000000000000000111101100\n25         T512  19341  127 00000000000000000000001000000000\n26         T514    594    3 00000000000000000000001000000010\n27         T917   4640   32 00000000000000000000001110010101\n28        T1132     16   16 00000000000000000000010001101100\n29        T1152      4    1 00000000000000000000010010000000\n30        T1310    179    1 00000000000000000000010100011110\n31        T1372  19491  104 00000000000000000000010101011100\n32        T2516   6640   32 00000000000000000000100111010100\n33        T6553   6838   76 00000000000000000001100110011001\n34        T8768     19   19 00000000000000000010001001000000\n35        T9174   8336   48 00000000000000000010001111010110\n36       T16384   3705   15 00000000000000000100000000000000\n37       T16512    198    1 00000000000000000100000010000000\n38       T18436   2574   13 00000000000000000100100000000100\n39       T26808   1368    9 00000000000000000110100010111000\n40       T32896    209    1 00000000000000001000000010000000\n41       T36992    211    1 00000000000000001001000010000000\n42       T41944   4880   48 00000000000000001010001111011000\n43       T48576  10457   56 00000000000000001011110111000000\n44       T55424      1    1 00000000000000001101100010000000\n45       T63616    152    1 00000000000000001111100010000000\n46       T65536   2532   18 00000000000000010000000000000000\n47       T83888  17835  150 00000000000000010100011110110000\n48       T86752     16   16 00000000000000010101001011100000\n49       T91764     16   16 00000000000000010110011001110100\n50       T96112     16   16 00000000000000010111011101110000\n51      T113776   5600   32 00000000000000011011110001110000\n52      T178258     13   13 00000000000000101011100001010010\n53      T262144  16167  118 00000000000001000000000000000000\n54      T331648    984    8 00000000000001010000111110000000\n55      T524288  25312  112 00000000000010000000000000000000\n56      T524544   2432   16 00000000000010000000000100000000\n57      T589824      9    9 00000000000010010000000000000000\n58      T618624   7030   57 00000000000010010111000010000000\n59      T678864     66   66 00000000000010100101101111010000\n60      T748928   2888   19 00000000000010110110110110000000\n61      T786432   7984   80 00000000000011000000000000000000\n62      T786944   2432   16 00000000000011000000001000000000\n63     T4718592   2464   48 00000000010010000000000000000000\n64     T8389632  10374   76 00000000100000000000010000000000\n65     T8651776   3781   38 00000000100001000000010000000000\n66     T8912896   2144   48 00000000100010000000000000000000\n67    T11141888   4160   32 00000000101010100000001100000000\n68    T12582912  14388   69 00000000110000000000000000000000\n69    T16777216   1251    9 00000001000000000000000000000000\n70    T16777232   6008   41 00000001000000000000000000010000\n71    T16777248   5200   26 00000001000000000000000000100000\n72    T16777472     13   13 00000001000000000000000100000000\n73    T16777557   7105   56 00000001000000000000000101010101\n74    T16779428     13   13 00000001000000000000100010100100\n75   T-18748192    153    2 00000001000111100001001100100000\n76   T-21474819    235    1 00000001010001111010111000000011\n77   T-21474825    113    1 00000001010001111010111000001001\n78   T-21474872     19    1 00000001010001111010111000111000\n79    T25165824     16   16 00000001100000000000000000000000\n80    T33554432     26   26 00000010000000000000000000000000\n81    T33554448   3616   41 00000010000000000000000000010000\n82    T33554944     13   13 00000010000000000000001000000000\n83   T134217728  17977   93 00001000000000000000000000000000\n84   T268435456  14934   66 00010000000000000000000000000000\n85 T-2147481664    204    1 01111111111111111111100001000000\n86 T-2147482176      1    1 01111111111111111111101001000000\n87 T-2147482432    210    1 01111111111111111111101101000000\n88 T-2147482688    208    1 01111111111111111111110001000000\n89 T-2147482816    212    1 01111111111111111111110011000000\n90 T-2147482944      1    1 01111111111111111111110101000000\n91 T-2147483646    152    1 01111111111111111111111111111110\n92 T-2147483647    152    1 01111111111111111111111111111111\n93 T-2147483648  12040   68 10000000000000000000000000000000\n94           NA 173966 1977 \n</code></pre>",
      "rawMarkdown": "LOL, I noticed those 2^x values early on. I think it is an artifact of how they coded these variables (see below). There are only 93 individual values, and most columns use only one value. T1 is the most common value. I think the real question is whether T1 means the same thing for each column? Also, is there something linking columns that use value T4?\r\n\r\n32 bit binary representation. In this table, Value is the character variable value. Total is the count of times that value appears in the character data table. Cols is the number of columns where that variable appears at least once. Binary is a binary representation of the numerical value. I used the absolute value of negative numbers. \r\n\r\n\r\n              Value  Total Cols                           Binary\r\n    1            T1 245319 1319 00000000000000000000000000000001\r\n    2            T2  25463  121 00000000000000000000000000000010\r\n    3            T3   5461   30 00000000000000000000000000000011\r\n    4            T4  14137  114 00000000000000000000000000000100\r\n    5            T5   2945   16 00000000000000000000000000000101\r\n    6            T6    964    4 00000000000000000000000000000110\r\n    7            T7    624    4 00000000000000000000000000000111\r\n    8            T8   9687   49 00000000000000000000000000001000\r\n    9            T9    354    2 00000000000000000000000000001001\r\n    10          T16  18768  110 00000000000000000000000000010000\r\n    11          T24   3060   21 00000000000000000000000000011000\r\n    12          T32  13641   91 00000000000000000000000000100000\r\n    13          T48   6135   37 00000000000000000000000000110000\r\n    14          T52   3406   13 00000000000000000000000000110100\r\n    15          T56   3211   13 00000000000000000000000000111000\r\n    16          T64   9307   51 00000000000000000000000001000000\r\n    17          T96   5205   36 00000000000000000000000001100000\r\n    18          T97    239    1 00000000000000000000000001100001\r\n    19          T98    530    2 00000000000000000000000001100010\r\n    20         T128   8280   36 00000000000000000000000010000000\r\n    21         T143    418    3 00000000000000000000000010001111\r\n    22         T145    437    3 00000000000000000000000010010001\r\n    23         T256   1338    9 00000000000000000000000100000000\r\n    24         T492    217    1 00000000000000000000000111101100\r\n    25         T512  19341  127 00000000000000000000001000000000\r\n    26         T514    594    3 00000000000000000000001000000010\r\n    27         T917   4640   32 00000000000000000000001110010101\r\n    28        T1132     16   16 00000000000000000000010001101100\r\n    29        T1152      4    1 00000000000000000000010010000000\r\n    30        T1310    179    1 00000000000000000000010100011110\r\n    31        T1372  19491  104 00000000000000000000010101011100\r\n    32        T2516   6640   32 00000000000000000000100111010100\r\n    33        T6553   6838   76 00000000000000000001100110011001\r\n    34        T8768     19   19 00000000000000000010001001000000\r\n    35        T9174   8336   48 00000000000000000010001111010110\r\n    36       T16384   3705   15 00000000000000000100000000000000\r\n    37       T16512    198    1 00000000000000000100000010000000\r\n    38       T18436   2574   13 00000000000000000100100000000100\r\n    39       T26808   1368    9 00000000000000000110100010111000\r\n    40       T32896    209    1 00000000000000001000000010000000\r\n    41       T36992    211    1 00000000000000001001000010000000\r\n    42       T41944   4880   48 00000000000000001010001111011000\r\n    43       T48576  10457   56 00000000000000001011110111000000\r\n    44       T55424      1    1 00000000000000001101100010000000\r\n    45       T63616    152    1 00000000000000001111100010000000\r\n    46       T65536   2532   18 00000000000000010000000000000000\r\n    47       T83888  17835  150 00000000000000010100011110110000\r\n    48       T86752     16   16 00000000000000010101001011100000\r\n    49       T91764     16   16 00000000000000010110011001110100\r\n    50       T96112     16   16 00000000000000010111011101110000\r\n    51      T113776   5600   32 00000000000000011011110001110000\r\n    52      T178258     13   13 00000000000000101011100001010010\r\n    53      T262144  16167  118 00000000000001000000000000000000\r\n    54      T331648    984    8 00000000000001010000111110000000\r\n    55      T524288  25312  112 00000000000010000000000000000000\r\n    56      T524544   2432   16 00000000000010000000000100000000\r\n    57      T589824      9    9 00000000000010010000000000000000\r\n    58      T618624   7030   57 00000000000010010111000010000000\r\n    59      T678864     66   66 00000000000010100101101111010000\r\n    60      T748928   2888   19 00000000000010110110110110000000\r\n    61      T786432   7984   80 00000000000011000000000000000000\r\n    62      T786944   2432   16 00000000000011000000001000000000\r\n    63     T4718592   2464   48 00000000010010000000000000000000\r\n    64     T8389632  10374   76 00000000100000000000010000000000\r\n    65     T8651776   3781   38 00000000100001000000010000000000\r\n    66     T8912896   2144   48 00000000100010000000000000000000\r\n    67    T11141888   4160   32 00000000101010100000001100000000\r\n    68    T12582912  14388   69 00000000110000000000000000000000\r\n    69    T16777216   1251    9 00000001000000000000000000000000\r\n    70    T16777232   6008   41 00000001000000000000000000010000\r\n    71    T16777248   5200   26 00000001000000000000000000100000\r\n    72    T16777472     13   13 00000001000000000000000100000000\r\n    73    T16777557   7105   56 00000001000000000000000101010101\r\n    74    T16779428     13   13 00000001000000000000100010100100\r\n    75   T-18748192    153    2 00000001000111100001001100100000\r\n    76   T-21474819    235    1 00000001010001111010111000000011\r\n    77   T-21474825    113    1 00000001010001111010111000001001\r\n    78   T-21474872     19    1 00000001010001111010111000111000\r\n    79    T25165824     16   16 00000001100000000000000000000000\r\n    80    T33554432     26   26 00000010000000000000000000000000\r\n    81    T33554448   3616   41 00000010000000000000000000010000\r\n    82    T33554944     13   13 00000010000000000000001000000000\r\n    83   T134217728  17977   93 00001000000000000000000000000000\r\n    84   T268435456  14934   66 00010000000000000000000000000000\r\n    85 T-2147481664    204    1 01111111111111111111100001000000\r\n    86 T-2147482176      1    1 01111111111111111111101001000000\r\n    87 T-2147482432    210    1 01111111111111111111101101000000\r\n    88 T-2147482688    208    1 01111111111111111111110001000000\r\n    89 T-2147482816    212    1 01111111111111111111110011000000\r\n    90 T-2147482944      1    1 01111111111111111111110101000000\r\n    91 T-2147483646    152    1 01111111111111111111111111111110\r\n    92 T-2147483647    152    1 01111111111111111111111111111111\r\n    93 T-2147483648  12040   68 10000000000000000000000000000000\r\n    94           NA 173966 1977",
      "votes": null
    },
    {
      "id": "137482",
      "postDate": "10/02/2016 01:03:31",
      "content": "<p>The negatives are from writing unsigned #'s as signed, so you should convert them back into uint32's before analysis.</p>",
      "rawMarkdown": "The negatives are from writing unsigned #'s as signed, so you should convert them back into uint32's before analysis.",
      "votes": null
    },
    {
      "id": "137486",
      "postDate": "10/02/2016 01:46:39",
      "content": "<p>This could be coincidence but the negative turn up at station 32--and the categories themselves at each step are very jumbled..... I haven't seen anything else to figure out WHAT the numbers are saying, but I'll leave this here--i think it may be related to the station somehow.  Having worked in mfg, I know that certain steps are &quot;Test&quot; steps and provide feedback to certain upstream processes or may deal with certain places on the piece.  In this case, it 'appears' (I use that cautiously, as it is only an appearance) that the category is somehow related to previous stations.</p>\n\n<pre><code>L0_S14_F361 {'T64', 'T96', 'T128', 'T32'}\nL0_S16_F425 {'T96', 'T2'}\nL0_S16_F430 {'T96', 'T2'}\nL0_S1_F31 {'T9', 'T24', 'T48'}\nL0_S21_F496 {'T4', 'T12582912', 'T48576', 'T16', 'T16777557', 'T512', 'T32', 'T16384', 'T33554448'}\nL0_S22_F575 {'T12582912', 'T48576', 'T16', 'T16777557', 'T512', 'T32', 'T8', 'T16777232'}\nL0_S2_F35 {'T64', 'T4', 'T32', 'T48', 'T96', 'T128'}\nL0_S2_F67 {'T64', 'T4', 'T32', 'T48', 'T96', 'T128'}\nL0_S4_F108 {'T97', 'T98'}\nL1_S24_F1206 {'T1372', 'T618624', 'T8389632', 'T83888'}\nL1_S24_F1365 {'T1372', 'T618624', 'T8389632', 'T8651776'}\nL1_S24_F1415 {'T33554432', 'T4', 'T134217728'}\nL1_S24_F1455 {'T33554432', 'T4', 'T134217728'}\nL1_S24_F1510 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F1525 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F1530 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F1671 {'T8768', 'T748928', 'T262144', 'T16', 'T618624', 'T24', 'T8389632', 'T83888', 'T1372', 'T8651776'}\nL1_S24_F1767 {'T331648', 'T16777216'}\nL1_S24_F1807 {'T331648', 'T16777216'}\nL1_S24_F675 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F695 {'T4', 'T2', 'T3'}\nL1_S24_F705 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F710 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F820 {'T512', 'T8389632', 'T1372'}\nL1_S24_F911 {'T1372', 'T512', 'T262144'}\nL1_S25_F1852 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\nL1_S25_F1897 {'T4', 'T5'}\nL1_S25_F2135 {'T4', 'T134217728'}\nL1_S25_F2229 {'T6', 'T4', 'T2', 'T3', 'T7', 'T1', 'T5'}\nL1_S25_F2404 {'T786432', 'T262144', 'T8912896', 'T524288', 'T41944', 'T83888'}\nL1_S25_F2496 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\nL1_S25_F2520 {'T524288', 'T83888'}\nL1_S25_F2671 {'T524288', 'T25165824', 'T83888'}\nL1_S25_F2676 {'T16', 'T134217728'}\nL1_S25_F2779 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\nL1_S25_F2803 {'T96112', 'T6553', 'T262144', 'T524288', 'T9174', 'T113776', 'T512', 'T786432', 'T41944'}\nL3_S32_F3854 {'T-2147482688', 'T63616', 'T36992', 'T2', 'T492', 'T55424', 'T48', 'T-2147482176', 'T16', 'T16512', 'T1', 'T-2147482816', 'T1152', 'T32896', 'T-2147483648', 'T256', 'T64', 'T-2147482432', 'T4', 'T1310', 'T128', 'T24', 'T512', 'T-21474872', 'T-2147481664', 'T32', 'T-2147482944', 'T8', 'T65536', 'T-21474825', 'T-21474819'}\nL3_S49_F4225 {'T16', 'T4', 'T2'}\nL3_S49_F4230 {'T16', 'T4', 'T2'}\nL3_S49_F4235 {'T16', 'T4', 'T2'}\nL3_S49_F4240 {'T16', 'T4', 'T2'}\n</code></pre>",
      "rawMarkdown": "This could be coincidence but the negative turn up at station 32--and the categories themselves at each step are very jumbled..... I haven't seen anything else to figure out WHAT the numbers are saying, but I'll leave this here--i think it may be related to the station somehow.  Having worked in mfg, I know that certain steps are \"Test\" steps and provide feedback to certain upstream processes or may deal with certain places on the piece.  In this case, it 'appears' (I use that cautiously, as it is only an appearance) that the category is somehow related to previous stations.\r\n\r\n  \r\n    L0_S14_F361 {'T64', 'T96', 'T128', 'T32'}\r\n    L0_S16_F425 {'T96', 'T2'}\r\n    L0_S16_F430 {'T96', 'T2'}\r\n    L0_S1_F31 {'T9', 'T24', 'T48'}\r\n    L0_S21_F496 {'T4', 'T12582912', 'T48576', 'T16', 'T16777557', 'T512', 'T32', 'T16384', 'T33554448'}\r\n    L0_S22_F575 {'T12582912', 'T48576', 'T16', 'T16777557', 'T512', 'T32', 'T8', 'T16777232'}\r\n    L0_S2_F35 {'T64', 'T4', 'T32', 'T48', 'T96', 'T128'}\r\n    L0_S2_F67 {'T64', 'T4', 'T32', 'T48', 'T96', 'T128'}\r\n    L0_S4_F108 {'T97', 'T98'}\r\n    L1_S24_F1206 {'T1372', 'T618624', 'T8389632', 'T83888'}\r\n    L1_S24_F1365 {'T1372', 'T618624', 'T8389632', 'T8651776'}\r\n    L1_S24_F1415 {'T33554432', 'T4', 'T134217728'}\r\n    L1_S24_F1455 {'T33554432', 'T4', 'T134217728'}\r\n    L1_S24_F1510 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F1525 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F1530 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F1671 {'T8768', 'T748928', 'T262144', 'T16', 'T618624', 'T24', 'T8389632', 'T83888', 'T1372', 'T8651776'}\r\n    L1_S24_F1767 {'T331648', 'T16777216'}\r\n    L1_S24_F1807 {'T331648', 'T16777216'}\r\n    L1_S24_F675 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F695 {'T4', 'T2', 'T3'}\r\n    L1_S24_F705 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F710 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F820 {'T512', 'T8389632', 'T1372'}\r\n    L1_S24_F911 {'T1372', 'T512', 'T262144'}\r\n    L1_S25_F1852 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\r\n    L1_S25_F1897 {'T4', 'T5'}\r\n    L1_S25_F2135 {'T4', 'T134217728'}\r\n    L1_S25_F2229 {'T6', 'T4', 'T2', 'T3', 'T7', 'T1', 'T5'}\r\n    L1_S25_F2404 {'T786432', 'T262144', 'T8912896', 'T524288', 'T41944', 'T83888'}\r\n    L1_S25_F2496 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\r\n    L1_S25_F2520 {'T524288', 'T83888'}\r\n    L1_S25_F2671 {'T524288', 'T25165824', 'T83888'}\r\n    L1_S25_F2676 {'T16', 'T134217728'}\r\n    L1_S25_F2779 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\r\n    L1_S25_F2803 {'T96112', 'T6553', 'T262144', 'T524288', 'T9174', 'T113776', 'T512', 'T786432', 'T41944'}\r\n    L3_S32_F3854 {'T-2147482688', 'T63616', 'T36992', 'T2', 'T492', 'T55424', 'T48', 'T-2147482176', 'T16', 'T16512', 'T1', 'T-2147482816', 'T1152', 'T32896', 'T-2147483648', 'T256', 'T64', 'T-2147482432', 'T4', 'T1310', 'T128', 'T24', 'T512', 'T-21474872', 'T-2147481664', 'T32', 'T-2147482944', 'T8', 'T65536', 'T-21474825', 'T-21474819'}\r\n    L3_S49_F4225 {'T16', 'T4', 'T2'}\r\n    L3_S49_F4230 {'T16', 'T4', 'T2'}\r\n    L3_S49_F4235 {'T16', 'T4', 'T2'}\r\n    L3_S49_F4240 {'T16', 'T4', 'T2'}",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 137473,
      "author_name": "rdslater",
      "author_url": "",
      "post_date": "10/01/2016 21:24:33",
      "content": "<p>How I looked at this:</p>\n\n<p>from my previous published kernel:</p>\n\n<pre><code>import pandas as pd\nimport numpy as np\nimport collections\ndata=pd.read_csv(&quot;./kaggle/Bosch/data/train_categorical.csv&quot;,chunksize=100000, dtype=str ,usecols=list(range(1,2141)))\nuniques = collections.defaultdict(set)\n\n\nfor chunk in data: \n    for col in chunk:\n        uniques[col] = uniques[col].union(chunk[col][chunk[col].notnull()].unique())\n</code></pre>\n\n<p>Then I looked at columns with more than 1 category:</p>\n\n<pre><code>empty_cols=[]\nsingle_cols=[]\nmulti_cols=[]\nfor key in uniques:\n    if len(uniques[key])==0:\n        empty_cols.append(key)\n    elif len(uniques[key])==1:\n        single_cols.append(key)\n    else:\n        multi_cols.append(key)\n</code></pre>\n\n<p>Then I put them all together:\ncats=[]</p>\n\n<pre><code>for item in multi_cols:\n    for thing in uniques[item]:\n        cats.append(thing)\nset(cats)\n</code></pre>\n\n<p>if you want a count of each column (just for giggles cause you like to stare at numbers like me:</p>\n\n<pre><code>cat_cnt=dict()\nfor item in cats:\n    if item in cat_cnt:\n        cat_cnt[item] += 1\n    else:\n        cat_cnt[item] = 1\n</code></pre>",
      "votes": null,
      "replies": []
    },
    {
      "id": 137476,
      "author_name": "raddar",
      "author_url": "",
      "post_date": "10/01/2016 21:35:54",
      "content": "<p>Have you made any use of it ?:) Interesting findings tho</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 137481,
      "author_name": "jeffhebert",
      "author_url": "",
      "post_date": "10/02/2016 00:57:04",
      "content": "<p>LOL, I noticed those 2^x values early on. I think it is an artifact of how they coded these variables (see below). There are only 93 individual values, and most columns use only one value. T1 is the most common value. I think the real question is whether T1 means the same thing for each column? Also, is there something linking columns that use value T4?</p>\n\n<p>32 bit binary representation. In this table, Value is the character variable value. Total is the count of times that value appears in the character data table. Cols is the number of columns where that variable appears at least once. Binary is a binary representation of the numerical value. I used the absolute value of negative numbers. </p>\n\n<pre><code>          Value  Total Cols                           Binary\n1            T1 245319 1319 00000000000000000000000000000001\n2            T2  25463  121 00000000000000000000000000000010\n3            T3   5461   30 00000000000000000000000000000011\n4            T4  14137  114 00000000000000000000000000000100\n5            T5   2945   16 00000000000000000000000000000101\n6            T6    964    4 00000000000000000000000000000110\n7            T7    624    4 00000000000000000000000000000111\n8            T8   9687   49 00000000000000000000000000001000\n9            T9    354    2 00000000000000000000000000001001\n10          T16  18768  110 00000000000000000000000000010000\n11          T24   3060   21 00000000000000000000000000011000\n12          T32  13641   91 00000000000000000000000000100000\n13          T48   6135   37 00000000000000000000000000110000\n14          T52   3406   13 00000000000000000000000000110100\n15          T56   3211   13 00000000000000000000000000111000\n16          T64   9307   51 00000000000000000000000001000000\n17          T96   5205   36 00000000000000000000000001100000\n18          T97    239    1 00000000000000000000000001100001\n19          T98    530    2 00000000000000000000000001100010\n20         T128   8280   36 00000000000000000000000010000000\n21         T143    418    3 00000000000000000000000010001111\n22         T145    437    3 00000000000000000000000010010001\n23         T256   1338    9 00000000000000000000000100000000\n24         T492    217    1 00000000000000000000000111101100\n25         T512  19341  127 00000000000000000000001000000000\n26         T514    594    3 00000000000000000000001000000010\n27         T917   4640   32 00000000000000000000001110010101\n28        T1132     16   16 00000000000000000000010001101100\n29        T1152      4    1 00000000000000000000010010000000\n30        T1310    179    1 00000000000000000000010100011110\n31        T1372  19491  104 00000000000000000000010101011100\n32        T2516   6640   32 00000000000000000000100111010100\n33        T6553   6838   76 00000000000000000001100110011001\n34        T8768     19   19 00000000000000000010001001000000\n35        T9174   8336   48 00000000000000000010001111010110\n36       T16384   3705   15 00000000000000000100000000000000\n37       T16512    198    1 00000000000000000100000010000000\n38       T18436   2574   13 00000000000000000100100000000100\n39       T26808   1368    9 00000000000000000110100010111000\n40       T32896    209    1 00000000000000001000000010000000\n41       T36992    211    1 00000000000000001001000010000000\n42       T41944   4880   48 00000000000000001010001111011000\n43       T48576  10457   56 00000000000000001011110111000000\n44       T55424      1    1 00000000000000001101100010000000\n45       T63616    152    1 00000000000000001111100010000000\n46       T65536   2532   18 00000000000000010000000000000000\n47       T83888  17835  150 00000000000000010100011110110000\n48       T86752     16   16 00000000000000010101001011100000\n49       T91764     16   16 00000000000000010110011001110100\n50       T96112     16   16 00000000000000010111011101110000\n51      T113776   5600   32 00000000000000011011110001110000\n52      T178258     13   13 00000000000000101011100001010010\n53      T262144  16167  118 00000000000001000000000000000000\n54      T331648    984    8 00000000000001010000111110000000\n55      T524288  25312  112 00000000000010000000000000000000\n56      T524544   2432   16 00000000000010000000000100000000\n57      T589824      9    9 00000000000010010000000000000000\n58      T618624   7030   57 00000000000010010111000010000000\n59      T678864     66   66 00000000000010100101101111010000\n60      T748928   2888   19 00000000000010110110110110000000\n61      T786432   7984   80 00000000000011000000000000000000\n62      T786944   2432   16 00000000000011000000001000000000\n63     T4718592   2464   48 00000000010010000000000000000000\n64     T8389632  10374   76 00000000100000000000010000000000\n65     T8651776   3781   38 00000000100001000000010000000000\n66     T8912896   2144   48 00000000100010000000000000000000\n67    T11141888   4160   32 00000000101010100000001100000000\n68    T12582912  14388   69 00000000110000000000000000000000\n69    T16777216   1251    9 00000001000000000000000000000000\n70    T16777232   6008   41 00000001000000000000000000010000\n71    T16777248   5200   26 00000001000000000000000000100000\n72    T16777472     13   13 00000001000000000000000100000000\n73    T16777557   7105   56 00000001000000000000000101010101\n74    T16779428     13   13 00000001000000000000100010100100\n75   T-18748192    153    2 00000001000111100001001100100000\n76   T-21474819    235    1 00000001010001111010111000000011\n77   T-21474825    113    1 00000001010001111010111000001001\n78   T-21474872     19    1 00000001010001111010111000111000\n79    T25165824     16   16 00000001100000000000000000000000\n80    T33554432     26   26 00000010000000000000000000000000\n81    T33554448   3616   41 00000010000000000000000000010000\n82    T33554944     13   13 00000010000000000000001000000000\n83   T134217728  17977   93 00001000000000000000000000000000\n84   T268435456  14934   66 00010000000000000000000000000000\n85 T-2147481664    204    1 01111111111111111111100001000000\n86 T-2147482176      1    1 01111111111111111111101001000000\n87 T-2147482432    210    1 01111111111111111111101101000000\n88 T-2147482688    208    1 01111111111111111111110001000000\n89 T-2147482816    212    1 01111111111111111111110011000000\n90 T-2147482944      1    1 01111111111111111111110101000000\n91 T-2147483646    152    1 01111111111111111111111111111110\n92 T-2147483647    152    1 01111111111111111111111111111111\n93 T-2147483648  12040   68 10000000000000000000000000000000\n94           NA 173966 1977 \n</code></pre>",
      "votes": null,
      "replies": []
    },
    {
      "id": 137482,
      "author_name": "happycube",
      "author_url": "",
      "post_date": "10/02/2016 01:03:31",
      "content": "<p>The negatives are from writing unsigned #'s as signed, so you should convert them back into uint32's before analysis.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 137486,
      "author_name": "rdslater",
      "author_url": "",
      "post_date": "10/02/2016 01:46:39",
      "content": "<p>This could be coincidence but the negative turn up at station 32--and the categories themselves at each step are very jumbled..... I haven't seen anything else to figure out WHAT the numbers are saying, but I'll leave this here--i think it may be related to the station somehow.  Having worked in mfg, I know that certain steps are &quot;Test&quot; steps and provide feedback to certain upstream processes or may deal with certain places on the piece.  In this case, it 'appears' (I use that cautiously, as it is only an appearance) that the category is somehow related to previous stations.</p>\n\n<pre><code>L0_S14_F361 {'T64', 'T96', 'T128', 'T32'}\nL0_S16_F425 {'T96', 'T2'}\nL0_S16_F430 {'T96', 'T2'}\nL0_S1_F31 {'T9', 'T24', 'T48'}\nL0_S21_F496 {'T4', 'T12582912', 'T48576', 'T16', 'T16777557', 'T512', 'T32', 'T16384', 'T33554448'}\nL0_S22_F575 {'T12582912', 'T48576', 'T16', 'T16777557', 'T512', 'T32', 'T8', 'T16777232'}\nL0_S2_F35 {'T64', 'T4', 'T32', 'T48', 'T96', 'T128'}\nL0_S2_F67 {'T64', 'T4', 'T32', 'T48', 'T96', 'T128'}\nL0_S4_F108 {'T97', 'T98'}\nL1_S24_F1206 {'T1372', 'T618624', 'T8389632', 'T83888'}\nL1_S24_F1365 {'T1372', 'T618624', 'T8389632', 'T8651776'}\nL1_S24_F1415 {'T33554432', 'T4', 'T134217728'}\nL1_S24_F1455 {'T33554432', 'T4', 'T134217728'}\nL1_S24_F1510 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F1525 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F1530 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F1671 {'T8768', 'T748928', 'T262144', 'T16', 'T618624', 'T24', 'T8389632', 'T83888', 'T1372', 'T8651776'}\nL1_S24_F1767 {'T331648', 'T16777216'}\nL1_S24_F1807 {'T331648', 'T16777216'}\nL1_S24_F675 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F695 {'T4', 'T2', 'T3'}\nL1_S24_F705 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F710 {'T4', 'T2', 'T5', 'T1', 'T3'}\nL1_S24_F820 {'T512', 'T8389632', 'T1372'}\nL1_S24_F911 {'T1372', 'T512', 'T262144'}\nL1_S25_F1852 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\nL1_S25_F1897 {'T4', 'T5'}\nL1_S25_F2135 {'T4', 'T134217728'}\nL1_S25_F2229 {'T6', 'T4', 'T2', 'T3', 'T7', 'T1', 'T5'}\nL1_S25_F2404 {'T786432', 'T262144', 'T8912896', 'T524288', 'T41944', 'T83888'}\nL1_S25_F2496 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\nL1_S25_F2520 {'T524288', 'T83888'}\nL1_S25_F2671 {'T524288', 'T25165824', 'T83888'}\nL1_S25_F2676 {'T16', 'T134217728'}\nL1_S25_F2779 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\nL1_S25_F2803 {'T96112', 'T6553', 'T262144', 'T524288', 'T9174', 'T113776', 'T512', 'T786432', 'T41944'}\nL3_S32_F3854 {'T-2147482688', 'T63616', 'T36992', 'T2', 'T492', 'T55424', 'T48', 'T-2147482176', 'T16', 'T16512', 'T1', 'T-2147482816', 'T1152', 'T32896', 'T-2147483648', 'T256', 'T64', 'T-2147482432', 'T4', 'T1310', 'T128', 'T24', 'T512', 'T-21474872', 'T-2147481664', 'T32', 'T-2147482944', 'T8', 'T65536', 'T-21474825', 'T-21474819'}\nL3_S49_F4225 {'T16', 'T4', 'T2'}\nL3_S49_F4230 {'T16', 'T4', 'T2'}\nL3_S49_F4235 {'T16', 'T4', 'T2'}\nL3_S49_F4240 {'T16', 'T4', 'T2'}\n</code></pre>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "137471": "So I was looking through my categorical data and it sure seemed like powers of two were popping up (a lot).\r\n\r\nLooking through my categorical data with multicolumns we have T2, T4, T32, T64, T128.  There seems to never be numbers in between them.  So i wasting my afternoon looking at this and thinking, those category labels cannot be arbitrary.  Some of them of silly long which seems ineffcient.  So what do they mean?\r\n\r\nI'm not sure just yet, but I found something very interesting.  It appears a large number of them are related to powers of 2.  I had calculated all the prime factors of the categories and it seemed like waaay to many were nice even powers of 2, very few were powers of 5 and many of them were very close to large powers of 2.   For example we have T33554432, T33554448, and T33554944.  Well the first one is 2^25 (yes i am a total math/physics nerd).   And the second one is just 16 more, but it factors kinda strange, but what the number is actualy (2^25+2^4)  the final one is (2^25+2^9)\r\n\r\nNow I first found this in the negative numbers--those struck me as really odd.  Who makes a category T-25151251?  Seems either inconsistent or it means something.  Well the most negative of those categories is -(2^31)=-2147483648.  Once again we can construct the near numbers through simple powers of 2. (-2147482944, -2147482816, -2147482688, -2147482432, -2147482176,-2147481664)\r\n\r\nAs to WHAT it actually means--I am not sure--this borders on data dredging, but these categories seem to be more than just labels.  There appears to be some sort of numeric information in them.\r\n\r\nThe following pure powers of 2 appear directly appear as categories: (2^N where N is):\r\n0,1,2,3,4,5,6,7,8,9,14,16,18,19,24,25,27,28,(-31).\r\n\r\nSomeone owes me a beer or something =)",
    "137473": "How I looked at this:\r\n\r\nfrom my previous published kernel:\r\n\r\n    import pandas as pd\r\n    import numpy as np\r\n    import collections\r\n    data=pd.read_csv(\"./kaggle/Bosch/data/train_categorical.csv\",chunksize=100000, dtype=str ,usecols=list(range(1,2141)))\r\n    uniques = collections.defaultdict(set)\r\n   \r\n  \r\n    for chunk in data: \r\n        for col in chunk:\r\n            uniques[col] = uniques[col].union(chunk[col][chunk[col].notnull()].unique())\r\n\r\nThen I looked at columns with more than 1 category:\r\n\r\n    empty_cols=[]\r\n    single_cols=[]\r\n    multi_cols=[]\r\n    for key in uniques:\r\n        if len(uniques[key])==0:\r\n            empty_cols.append(key)\r\n        elif len(uniques[key])==1:\r\n            single_cols.append(key)\r\n        else:\r\n            multi_cols.append(key)\r\nThen I put them all together:\r\ncats=[]\r\n\r\n    for item in multi_cols:\r\n        for thing in uniques[item]:\r\n            cats.append(thing)\r\n    set(cats)\r\n\r\nif you want a count of each column (just for giggles cause you like to stare at numbers like me:\r\n\r\n    cat_cnt=dict()\r\n    for item in cats:\r\n        if item in cat_cnt:\r\n            cat_cnt[item] += 1\r\n        else:\r\n            cat_cnt[item] = 1",
    "137476": "Have you made any use of it ?:) Interesting findings tho",
    "137481": "LOL, I noticed those 2^x values early on. I think it is an artifact of how they coded these variables (see below). There are only 93 individual values, and most columns use only one value. T1 is the most common value. I think the real question is whether T1 means the same thing for each column? Also, is there something linking columns that use value T4?\r\n\r\n32 bit binary representation. In this table, Value is the character variable value. Total is the count of times that value appears in the character data table. Cols is the number of columns where that variable appears at least once. Binary is a binary representation of the numerical value. I used the absolute value of negative numbers. \r\n\r\n\r\n              Value  Total Cols                           Binary\r\n    1            T1 245319 1319 00000000000000000000000000000001\r\n    2            T2  25463  121 00000000000000000000000000000010\r\n    3            T3   5461   30 00000000000000000000000000000011\r\n    4            T4  14137  114 00000000000000000000000000000100\r\n    5            T5   2945   16 00000000000000000000000000000101\r\n    6            T6    964    4 00000000000000000000000000000110\r\n    7            T7    624    4 00000000000000000000000000000111\r\n    8            T8   9687   49 00000000000000000000000000001000\r\n    9            T9    354    2 00000000000000000000000000001001\r\n    10          T16  18768  110 00000000000000000000000000010000\r\n    11          T24   3060   21 00000000000000000000000000011000\r\n    12          T32  13641   91 00000000000000000000000000100000\r\n    13          T48   6135   37 00000000000000000000000000110000\r\n    14          T52   3406   13 00000000000000000000000000110100\r\n    15          T56   3211   13 00000000000000000000000000111000\r\n    16          T64   9307   51 00000000000000000000000001000000\r\n    17          T96   5205   36 00000000000000000000000001100000\r\n    18          T97    239    1 00000000000000000000000001100001\r\n    19          T98    530    2 00000000000000000000000001100010\r\n    20         T128   8280   36 00000000000000000000000010000000\r\n    21         T143    418    3 00000000000000000000000010001111\r\n    22         T145    437    3 00000000000000000000000010010001\r\n    23         T256   1338    9 00000000000000000000000100000000\r\n    24         T492    217    1 00000000000000000000000111101100\r\n    25         T512  19341  127 00000000000000000000001000000000\r\n    26         T514    594    3 00000000000000000000001000000010\r\n    27         T917   4640   32 00000000000000000000001110010101\r\n    28        T1132     16   16 00000000000000000000010001101100\r\n    29        T1152      4    1 00000000000000000000010010000000\r\n    30        T1310    179    1 00000000000000000000010100011110\r\n    31        T1372  19491  104 00000000000000000000010101011100\r\n    32        T2516   6640   32 00000000000000000000100111010100\r\n    33        T6553   6838   76 00000000000000000001100110011001\r\n    34        T8768     19   19 00000000000000000010001001000000\r\n    35        T9174   8336   48 00000000000000000010001111010110\r\n    36       T16384   3705   15 00000000000000000100000000000000\r\n    37       T16512    198    1 00000000000000000100000010000000\r\n    38       T18436   2574   13 00000000000000000100100000000100\r\n    39       T26808   1368    9 00000000000000000110100010111000\r\n    40       T32896    209    1 00000000000000001000000010000000\r\n    41       T36992    211    1 00000000000000001001000010000000\r\n    42       T41944   4880   48 00000000000000001010001111011000\r\n    43       T48576  10457   56 00000000000000001011110111000000\r\n    44       T55424      1    1 00000000000000001101100010000000\r\n    45       T63616    152    1 00000000000000001111100010000000\r\n    46       T65536   2532   18 00000000000000010000000000000000\r\n    47       T83888  17835  150 00000000000000010100011110110000\r\n    48       T86752     16   16 00000000000000010101001011100000\r\n    49       T91764     16   16 00000000000000010110011001110100\r\n    50       T96112     16   16 00000000000000010111011101110000\r\n    51      T113776   5600   32 00000000000000011011110001110000\r\n    52      T178258     13   13 00000000000000101011100001010010\r\n    53      T262144  16167  118 00000000000001000000000000000000\r\n    54      T331648    984    8 00000000000001010000111110000000\r\n    55      T524288  25312  112 00000000000010000000000000000000\r\n    56      T524544   2432   16 00000000000010000000000100000000\r\n    57      T589824      9    9 00000000000010010000000000000000\r\n    58      T618624   7030   57 00000000000010010111000010000000\r\n    59      T678864     66   66 00000000000010100101101111010000\r\n    60      T748928   2888   19 00000000000010110110110110000000\r\n    61      T786432   7984   80 00000000000011000000000000000000\r\n    62      T786944   2432   16 00000000000011000000001000000000\r\n    63     T4718592   2464   48 00000000010010000000000000000000\r\n    64     T8389632  10374   76 00000000100000000000010000000000\r\n    65     T8651776   3781   38 00000000100001000000010000000000\r\n    66     T8912896   2144   48 00000000100010000000000000000000\r\n    67    T11141888   4160   32 00000000101010100000001100000000\r\n    68    T12582912  14388   69 00000000110000000000000000000000\r\n    69    T16777216   1251    9 00000001000000000000000000000000\r\n    70    T16777232   6008   41 00000001000000000000000000010000\r\n    71    T16777248   5200   26 00000001000000000000000000100000\r\n    72    T16777472     13   13 00000001000000000000000100000000\r\n    73    T16777557   7105   56 00000001000000000000000101010101\r\n    74    T16779428     13   13 00000001000000000000100010100100\r\n    75   T-18748192    153    2 00000001000111100001001100100000\r\n    76   T-21474819    235    1 00000001010001111010111000000011\r\n    77   T-21474825    113    1 00000001010001111010111000001001\r\n    78   T-21474872     19    1 00000001010001111010111000111000\r\n    79    T25165824     16   16 00000001100000000000000000000000\r\n    80    T33554432     26   26 00000010000000000000000000000000\r\n    81    T33554448   3616   41 00000010000000000000000000010000\r\n    82    T33554944     13   13 00000010000000000000001000000000\r\n    83   T134217728  17977   93 00001000000000000000000000000000\r\n    84   T268435456  14934   66 00010000000000000000000000000000\r\n    85 T-2147481664    204    1 01111111111111111111100001000000\r\n    86 T-2147482176      1    1 01111111111111111111101001000000\r\n    87 T-2147482432    210    1 01111111111111111111101101000000\r\n    88 T-2147482688    208    1 01111111111111111111110001000000\r\n    89 T-2147482816    212    1 01111111111111111111110011000000\r\n    90 T-2147482944      1    1 01111111111111111111110101000000\r\n    91 T-2147483646    152    1 01111111111111111111111111111110\r\n    92 T-2147483647    152    1 01111111111111111111111111111111\r\n    93 T-2147483648  12040   68 10000000000000000000000000000000\r\n    94           NA 173966 1977",
    "137482": "The negatives are from writing unsigned #'s as signed, so you should convert them back into uint32's before analysis.",
    "137486": "This could be coincidence but the negative turn up at station 32--and the categories themselves at each step are very jumbled..... I haven't seen anything else to figure out WHAT the numbers are saying, but I'll leave this here--i think it may be related to the station somehow.  Having worked in mfg, I know that certain steps are \"Test\" steps and provide feedback to certain upstream processes or may deal with certain places on the piece.  In this case, it 'appears' (I use that cautiously, as it is only an appearance) that the category is somehow related to previous stations.\r\n\r\n  \r\n    L0_S14_F361 {'T64', 'T96', 'T128', 'T32'}\r\n    L0_S16_F425 {'T96', 'T2'}\r\n    L0_S16_F430 {'T96', 'T2'}\r\n    L0_S1_F31 {'T9', 'T24', 'T48'}\r\n    L0_S21_F496 {'T4', 'T12582912', 'T48576', 'T16', 'T16777557', 'T512', 'T32', 'T16384', 'T33554448'}\r\n    L0_S22_F575 {'T12582912', 'T48576', 'T16', 'T16777557', 'T512', 'T32', 'T8', 'T16777232'}\r\n    L0_S2_F35 {'T64', 'T4', 'T32', 'T48', 'T96', 'T128'}\r\n    L0_S2_F67 {'T64', 'T4', 'T32', 'T48', 'T96', 'T128'}\r\n    L0_S4_F108 {'T97', 'T98'}\r\n    L1_S24_F1206 {'T1372', 'T618624', 'T8389632', 'T83888'}\r\n    L1_S24_F1365 {'T1372', 'T618624', 'T8389632', 'T8651776'}\r\n    L1_S24_F1415 {'T33554432', 'T4', 'T134217728'}\r\n    L1_S24_F1455 {'T33554432', 'T4', 'T134217728'}\r\n    L1_S24_F1510 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F1525 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F1530 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F1671 {'T8768', 'T748928', 'T262144', 'T16', 'T618624', 'T24', 'T8389632', 'T83888', 'T1372', 'T8651776'}\r\n    L1_S24_F1767 {'T331648', 'T16777216'}\r\n    L1_S24_F1807 {'T331648', 'T16777216'}\r\n    L1_S24_F675 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F695 {'T4', 'T2', 'T3'}\r\n    L1_S24_F705 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F710 {'T4', 'T2', 'T5', 'T1', 'T3'}\r\n    L1_S24_F820 {'T512', 'T8389632', 'T1372'}\r\n    L1_S24_F911 {'T1372', 'T512', 'T262144'}\r\n    L1_S25_F1852 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\r\n    L1_S25_F1897 {'T4', 'T5'}\r\n    L1_S25_F2135 {'T4', 'T134217728'}\r\n    L1_S25_F2229 {'T6', 'T4', 'T2', 'T3', 'T7', 'T1', 'T5'}\r\n    L1_S25_F2404 {'T786432', 'T262144', 'T8912896', 'T524288', 'T41944', 'T83888'}\r\n    L1_S25_F2496 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\r\n    L1_S25_F2520 {'T524288', 'T83888'}\r\n    L1_S25_F2671 {'T524288', 'T25165824', 'T83888'}\r\n    L1_S25_F2676 {'T16', 'T134217728'}\r\n    L1_S25_F2779 {'T4', 'T2', 'T3', 'T1', 'T6', 'T8', 'T7', 'T5'}\r\n    L1_S25_F2803 {'T96112', 'T6553', 'T262144', 'T524288', 'T9174', 'T113776', 'T512', 'T786432', 'T41944'}\r\n    L3_S32_F3854 {'T-2147482688', 'T63616', 'T36992', 'T2', 'T492', 'T55424', 'T48', 'T-2147482176', 'T16', 'T16512', 'T1', 'T-2147482816', 'T1152', 'T32896', 'T-2147483648', 'T256', 'T64', 'T-2147482432', 'T4', 'T1310', 'T128', 'T24', 'T512', 'T-21474872', 'T-2147481664', 'T32', 'T-2147482944', 'T8', 'T65536', 'T-21474825', 'T-21474819'}\r\n    L3_S49_F4225 {'T16', 'T4', 'T2'}\r\n    L3_S49_F4230 {'T16', 'T4', 'T2'}\r\n    L3_S49_F4235 {'T16', 'T4', 'T2'}\r\n    L3_S49_F4240 {'T16', 'T4', 'T2'}"
  },
  "source": "meta"
}