{
  "id": 21966,
  "title": "Neural Network using Keras",
  "url": "/competitions/grupo-bimbo-inventory-demand/discussion/21966",
  "author_name": "alex mulo",
  "post_date": "2016-06-29T21:09:09.347000",
  "votes": 1,
  "comment_count": 13,
  "views": 2926,
  "content": "<p>Hello guys,</p>\n\n<p>I'd like to use a neural network using Keras, but I'm not sure about the code to use to set the model. I did this:</p>\n\n<pre><code>model = Sequential()\nmodel.add(Dense(100,input_dim=X_train.shape[1]))\nmodel.add(Activation('linear'))\nmodel.add(Dense(1))\n</code></pre>\n\n<p>What do you think, could it work or would you suggest something else?</p>\n\n<p>Thanks!!!</p>",
  "messages": [
    {
      "id": 129146,
      "postDate": "2016-07-27T03:23:07.823Z",
      "content": "<p>[quote=poteman;128724]</p>\n\n<p>can you give me an simple example python code for solve this problem using keras?</p>\n\n<p>[/quote]</p>\n\n<p>Code that I use : You will see a lot of print statements, I am new to Python, hence debug using prints.</p>\n\n<p>for e in range(20): \n    print(e)</p>\n\n<p>for file_ in files: ## you can ignore this, I had split up the file in weeks for some data processing, hence reading all files in a directory. Not required if you have single train.csv</p>\n\n<pre><code>    reader =pd.read_csv(file_,chunksize=50000,usecols=fields,index_col=None)\n    for chunk in reader:\n        print(chunk.shape)\n\n        y=chunk.Demanda_uni_equil\n        print(y.shape)\n        chunk.drop(&quot;Demanda_uni_equil&quot;,inplace=True,axis=1)\n        print chunk.keys()#debug\n        print(chunk.shape)\n        chunk = chunk.as_matrix()#keras does not accept dataframes\n        print(type(chunk))\n        print(type(y))\n        print(e)\n        model.fit(chunk, y, verbose=1,nb_epoch=1)# batch_size=1000,shuffle=True,\nsave_model_wt(model)\n</code></pre>\n\n<p>Explanation: I train this in minibatches of 50000 rows, each with epoch=1, and then run the entire loop for 20 epochs.\nPlease note with 600MB GPU, on i3, 8GB machine, this is running for the past 1 week. It should end tomorrow by my estimates. I guess patience is a virtue.</p>\n\n<p>Regards</p>",
      "rawMarkdown": "[quote=poteman;128724]\r\n\r\ncan you give me an simple example python code for solve this problem using keras?\r\n\r\n[/quote]\r\n\r\nCode that I use : You will see a lot of print statements, I am new to Python, hence debug using prints.\r\n\r\nfor e in range(20): \r\n    print(e)\r\n    \r\nfor file_ in files: ## you can ignore this, I had split up the file in weeks for some data processing, hence reading all files in a directory. Not required if you have single train.csv\r\n        \r\n        reader =pd.read_csv(file_,chunksize=50000,usecols=fields,index_col=None)\r\n        for chunk in reader:\r\n            print(chunk.shape)\r\n\r\n            y=chunk.Demanda_uni_equil\r\n            print(y.shape)\r\n            chunk.drop(\"Demanda_uni_equil\",inplace=True,axis=1)\r\n            print chunk.keys()#debug\r\n            print(chunk.shape)\r\n            chunk = chunk.as_matrix()#keras does not accept dataframes\r\n            print(type(chunk))\r\n            print(type(y))\r\n            print(e)\r\n            model.fit(chunk, y, verbose=1,nb_epoch=1)# batch_size=1000,shuffle=True,\r\n    save_model_wt(model)\r\n\r\nExplanation: I train this in minibatches of 50000 rows, each with epoch=1, and then run the entire loop for 20 epochs.\r\nPlease note with 600MB GPU, on i3, 8GB machine, this is running for the past 1 week. It should end tomorrow by my estimates. I guess patience is a virtue.\r\n\r\nRegards\r\n\r\n\r\n",
      "votes": 1
    },
    {
      "id": 128724,
      "postDate": "2016-07-23T03:57:30.007Z",
      "content": "<p>can you give me an simple example python code for solve this problem using keras?</p>",
      "rawMarkdown": "can you give me an simple example python code for solve this problem using keras?",
      "votes": 1
    },
    {
      "id": 127641,
      "postDate": "2016-07-14T19:47:45.607Z",
      "content": "<p>zillerxi:  It's just like training a smaller data set, it merely takes a lot longer, so having a reasonable GPU is very important.  NN training usually occurs in batches of at most a few thousand.</p>",
      "rawMarkdown": "zillerxi:  It's just like training a smaller data set, it merely takes a lot longer, so having a reasonable GPU is very important.  NN training usually occurs in batches of at most a few thousand.",
      "votes": 1
    },
    {
      "id": 127104,
      "postDate": "2016-07-13T11:20:26.047Z",
      "content": "<p>Any specific reason for your activation function choices?</p>",
      "rawMarkdown": "Any specific reason for your activation function choices?",
      "votes": 1
    },
    {
      "id": 127083,
      "postDate": "2016-07-13T08:56:07.893Z",
      "content": "<p>[quote=Insaf Ashrapov;127069]</p>\n\n<p>@mgchbot what activation functions have you used, have you applied one-hot encoding?</p>\n\n<p>[/quote]</p>\n\n<p>1).RELU for hidden layers and linear activation for the output layer.\n2).Yes,one-hot encoding with Feature_hashing to save memory.</p>",
      "rawMarkdown": "[quote=Insaf Ashrapov;127069]\r\n\r\n@mgchbot what activation functions have you used, have you applied one-hot encoding?\r\n\r\n[/quote]\r\n\r\n1).RELU for hidden layers and linear activation for the output layer.\r\n2).Yes,one-hot encoding with Feature_hashing to save memory.",
      "votes": 1
    },
    {
      "id": 125479,
      "postDate": "2016-06-29T21:09:09.347Z",
      "content": "<p>Hello guys,</p>\n\n<p>I'd like to use a neural network using Keras, but I'm not sure about the code to use to set the model. I did this:</p>\n\n<pre><code>model = Sequential()\nmodel.add(Dense(100,input_dim=X_train.shape[1]))\nmodel.add(Activation('linear'))\nmodel.add(Dense(1))\n</code></pre>\n\n<p>What do you think, could it work or would you suggest something else?</p>\n\n<p>Thanks!!!</p>",
      "rawMarkdown": "Hello guys,\r\n\r\nI'd like to use a neural network using Keras, but I'm not sure about the code to use to set the model. I did this:\r\n\r\n    model = Sequential()\r\n    model.add(Dense(100,input_dim=X_train.shape[1]))\r\n    model.add(Activation('linear'))\r\n    model.add(Dense(1))\r\n\r\nWhat do you think, could it work or would you suggest something else?\r\n\r\nThanks!!!",
      "votes": 1
    },
    {
      "id": 128943,
      "postDate": "2016-07-25T11:26:55.623Z",
      "content": "<p>[quote=mgchbot;127083]</p>\n\n<p>[quote=Insaf Ashrapov;127069]</p>\n\n<p>@mgchbot what activation functions have you used, have you applied one-hot encoding?</p>\n\n<p>[/quote]</p>\n\n<p>1).RELU for hidden layers and linear activation for the output layer.\n2).Yes,one-hot encoding with Feature_hashing to save memory.</p>\n\n<p>[/quote]\ncan you give me an simple example python code for solve this problem using keras?</p>",
      "rawMarkdown": "[quote=mgchbot;127083]\r\n\r\n[quote=Insaf Ashrapov;127069]\r\n\r\n@mgchbot what activation functions have you used, have you applied one-hot encoding?\r\n\r\n[/quote]\r\n\r\n1).RELU for hidden layers and linear activation for the output layer.\r\n2).Yes,one-hot encoding with Feature_hashing to save memory.\r\n\r\n[/quote]\r\ncan you give me an simple example python code for solve this problem using keras?",
      "votes": 2
    },
    {
      "id": 126825,
      "postDate": "2016-07-12T13:56:05.577Z",
      "content": "<p>I got ~0.48 with a simple 3 layers feedforward Neural Network(1536*3),your code is right but the size of your Neural Network should be larger.</p>",
      "rawMarkdown": "I got ~0.48 with a simple 3 layers feedforward Neural Network(1536*3),your code is right but the size of your Neural Network should be larger.",
      "votes": 2
    },
    {
      "id": 129756,
      "postDate": "2016-08-02T05:10:37.920Z",
      "content": "<p>How are those of you using neural networks modeling the problem? Are you simply treating all the past data as a training set and just performing a mutlinomial regression? Unless one uses some sort of sequential model, the dynamical nature of the problem will be missed; I would expect the demand of a product to only be influenced by things happening locally (a sale/promotion, public opinion, other unexplainable phenomenon).</p>",
      "rawMarkdown": "How are those of you using neural networks modeling the problem? Are you simply treating all the past data as a training set and just performing a mutlinomial regression? Unless one uses some sort of sequential model, the dynamical nature of the problem will be missed; I would expect the demand of a product to only be influenced by things happening locally (a sale/promotion, public opinion, other unexplainable phenomenon)."
    },
    {
      "id": 128820,
      "postDate": "2016-07-24T10:52:06.760Z",
      "content": "<p>[quote=mgchbot;127083]</p>\n\n<p>[quote=Insaf Ashrapov;127069]</p>\n\n<p>@mgchbot what activation functions have you used, have you applied one-hot encoding?</p>\n\n<p>[/quote]</p>\n\n<p>1).RELU for hidden layers and linear activation for the output layer.\n2).Yes,one-hot encoding with Feature_hashing to save memory.</p>\n\n<p>[/quote]</p>\n\n<p>@mgchbot: what configuration do you have? Which graphic card do you use?</p>\n\n<p>Thanks</p>",
      "rawMarkdown": "[quote=mgchbot;127083]\r\n\r\n[quote=Insaf Ashrapov;127069]\r\n\r\n@mgchbot what activation functions have you used, have you applied one-hot encoding?\r\n\r\n[/quote]\r\n\r\n1).RELU for hidden layers and linear activation for the output layer.\r\n2).Yes,one-hot encoding with Feature_hashing to save memory.\r\n\r\n[/quote]\r\n\r\n@mgchbot: what configuration do you have? Which graphic card do you use?\r\n\r\nThanks"
    },
    {
      "id": 127426,
      "postDate": "2016-07-14T08:00:15.863Z",
      "content": "<p>I don't know how to train such a large data set on neural network. </p>",
      "rawMarkdown": "I don't know how to train such a large data set on neural network. "
    },
    {
      "id": 127106,
      "postDate": "2016-07-13T11:23:34.560Z",
      "content": "<p>How many samples did you use for the training? How long did it take to train. </p>\n\n<p>Tahnks!!</p>",
      "rawMarkdown": "How many samples did you use for the training? How long did it take to train. \r\n\r\nTahnks!!"
    },
    {
      "id": 127069,
      "postDate": "2016-07-13T08:03:08.650Z",
      "content": "<p>@mgchbot what activation functions have you used, have you applied one-hot encoding?</p>",
      "rawMarkdown": "@mgchbot what activation functions have you used, have you applied one-hot encoding?"
    },
    {
      "id": 129176,
      "postDate": "2016-07-27T09:45:45.687Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 129146,
      "author_name": "TheFOX",
      "author_url": "",
      "post_date": "2016-07-27T03:23:07.823000",
      "content": "<p>[quote=poteman;128724]</p>\n\n<p>can you give me an simple example python code for solve this problem using keras?</p>\n\n<p>[/quote]</p>\n\n<p>Code that I use : You will see a lot of print statements, I am new to Python, hence debug using prints.</p>\n\n<p>for e in range(20): \n    print(e)</p>\n\n<p>for file_ in files: ## you can ignore this, I had split up the file in weeks for some data processing, hence reading all files in a directory. Not required if you have single train.csv</p>\n\n<pre><code>    reader =pd.read_csv(file_,chunksize=50000,usecols=fields,index_col=None)\n    for chunk in reader:\n        print(chunk.shape)\n\n        y=chunk.Demanda_uni_equil\n        print(y.shape)\n        chunk.drop(&quot;Demanda_uni_equil&quot;,inplace=True,axis=1)\n        print chunk.keys()#debug\n        print(chunk.shape)\n        chunk = chunk.as_matrix()#keras does not accept dataframes\n        print(type(chunk))\n        print(type(y))\n        print(e)\n        model.fit(chunk, y, verbose=1,nb_epoch=1)# batch_size=1000,shuffle=True,\nsave_model_wt(model)\n</code></pre>\n\n<p>Explanation: I train this in minibatches of 50000 rows, each with epoch=1, and then run the entire loop for 20 epochs.\nPlease note with 600MB GPU, on i3, 8GB machine, this is running for the past 1 week. It should end tomorrow by my estimates. I guess patience is a virtue.</p>\n\n<p>Regards</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 128724,
      "author_name": "",
      "author_url": "",
      "post_date": "2016-07-23T03:57:30.007000",
      "content": "<p>can you give me an simple example python code for solve this problem using keras?</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 127641,
      "author_name": "happycube",
      "author_url": "",
      "post_date": "2016-07-14T19:47:45.607000",
      "content": "<p>zillerxi:  It's just like training a smaller data set, it merely takes a lot longer, so having a reasonable GPU is very important.  NN training usually occurs in batches of at most a few thousand.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 127104,
      "author_name": "Andrich van Wyk",
      "author_url": "",
      "post_date": "2016-07-13T11:20:26.047000",
      "content": "<p>Any specific reason for your activation function choices?</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 127083,
      "author_name": "mgchbot",
      "author_url": "",
      "post_date": "2016-07-13T08:56:07.893000",
      "content": "<p>[quote=Insaf Ashrapov;127069]</p>\n\n<p>@mgchbot what activation functions have you used, have you applied one-hot encoding?</p>\n\n<p>[/quote]</p>\n\n<p>1).RELU for hidden layers and linear activation for the output layer.\n2).Yes,one-hot encoding with Feature_hashing to save memory.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 128943,
      "author_name": "",
      "author_url": "",
      "post_date": "2016-07-25T11:26:55.623000",
      "content": "<p>[quote=mgchbot;127083]</p>\n\n<p>[quote=Insaf Ashrapov;127069]</p>\n\n<p>@mgchbot what activation functions have you used, have you applied one-hot encoding?</p>\n\n<p>[/quote]</p>\n\n<p>1).RELU for hidden layers and linear activation for the output layer.\n2).Yes,one-hot encoding with Feature_hashing to save memory.</p>\n\n<p>[/quote]\ncan you give me an simple example python code for solve this problem using keras?</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 126825,
      "author_name": "mgchbot",
      "author_url": "",
      "post_date": "2016-07-12T13:56:05.577000",
      "content": "<p>I got ~0.48 with a simple 3 layers feedforward Neural Network(1536*3),your code is right but the size of your Neural Network should be larger.</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 129756,
      "author_name": "Kirwi",
      "author_url": "",
      "post_date": "2016-08-02T05:10:37.920000",
      "content": "<p>How are those of you using neural networks modeling the problem? Are you simply treating all the past data as a training set and just performing a mutlinomial regression? Unless one uses some sort of sequential model, the dynamical nature of the problem will be missed; I would expect the demand of a product to only be influenced by things happening locally (a sale/promotion, public opinion, other unexplainable phenomenon).</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 128820,
      "author_name": "alex mulo",
      "author_url": "",
      "post_date": "2016-07-24T10:52:06.760000",
      "content": "<p>[quote=mgchbot;127083]</p>\n\n<p>[quote=Insaf Ashrapov;127069]</p>\n\n<p>@mgchbot what activation functions have you used, have you applied one-hot encoding?</p>\n\n<p>[/quote]</p>\n\n<p>1).RELU for hidden layers and linear activation for the output layer.\n2).Yes,one-hot encoding with Feature_hashing to save memory.</p>\n\n<p>[/quote]</p>\n\n<p>@mgchbot: what configuration do you have? Which graphic card do you use?</p>\n\n<p>Thanks</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 127426,
      "author_name": "zillerxi",
      "author_url": "",
      "post_date": "2016-07-14T08:00:15.863000",
      "content": "<p>I don't know how to train such a large data set on neural network. </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 127106,
      "author_name": "alex mulo",
      "author_url": "",
      "post_date": "2016-07-13T11:23:34.560000",
      "content": "<p>How many samples did you use for the training? How long did it take to train. </p>\n\n<p>Tahnks!!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 127069,
      "author_name": "Insaf Ashrapov",
      "author_url": "",
      "post_date": "2016-07-13T08:03:08.650000",
      "content": "<p>@mgchbot what activation functions have you used, have you applied one-hot encoding?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 129176,
      "author_name": "",
      "author_url": "",
      "post_date": "2016-07-27T09:45:45.687000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "129146": "[quote=poteman;128724]\r\n\r\ncan you give me an simple example python code for solve this problem using keras?\r\n\r\n[/quote]\r\n\r\nCode that I use : You will see a lot of print statements, I am new to Python, hence debug using prints.\r\n\r\nfor e in range(20): \r\n    print(e)\r\n    \r\nfor file_ in files: ## you can ignore this, I had split up the file in weeks for some data processing, hence reading all files in a directory. Not required if you have single train.csv\r\n        \r\n        reader =pd.read_csv(file_,chunksize=50000,usecols=fields,index_col=None)\r\n        for chunk in reader:\r\n            print(chunk.shape)\r\n\r\n            y=chunk.Demanda_uni_equil\r\n            print(y.shape)\r\n            chunk.drop(\"Demanda_uni_equil\",inplace=True,axis=1)\r\n            print chunk.keys()#debug\r\n            print(chunk.shape)\r\n            chunk = chunk.as_matrix()#keras does not accept dataframes\r\n            print(type(chunk))\r\n            print(type(y))\r\n            print(e)\r\n            model.fit(chunk, y, verbose=1,nb_epoch=1)# batch_size=1000,shuffle=True,\r\n    save_model_wt(model)\r\n\r\nExplanation: I train this in minibatches of 50000 rows, each with epoch=1, and then run the entire loop for 20 epochs.\r\nPlease note with 600MB GPU, on i3, 8GB machine, this is running for the past 1 week. It should end tomorrow by my estimates. I guess patience is a virtue.\r\n\r\nRegards\r\n\r\n\r\n",
    "128724": "can you give me an simple example python code for solve this problem using keras?",
    "127641": "zillerxi:  It's just like training a smaller data set, it merely takes a lot longer, so having a reasonable GPU is very important.  NN training usually occurs in batches of at most a few thousand.",
    "127104": "Any specific reason for your activation function choices?",
    "127083": "[quote=Insaf Ashrapov;127069]\r\n\r\n@mgchbot what activation functions have you used, have you applied one-hot encoding?\r\n\r\n[/quote]\r\n\r\n1).RELU for hidden layers and linear activation for the output layer.\r\n2).Yes,one-hot encoding with Feature_hashing to save memory.",
    "125479": "Hello guys,\r\n\r\nI'd like to use a neural network using Keras, but I'm not sure about the code to use to set the model. I did this:\r\n\r\n    model = Sequential()\r\n    model.add(Dense(100,input_dim=X_train.shape[1]))\r\n    model.add(Activation('linear'))\r\n    model.add(Dense(1))\r\n\r\nWhat do you think, could it work or would you suggest something else?\r\n\r\nThanks!!!",
    "128943": "[quote=mgchbot;127083]\r\n\r\n[quote=Insaf Ashrapov;127069]\r\n\r\n@mgchbot what activation functions have you used, have you applied one-hot encoding?\r\n\r\n[/quote]\r\n\r\n1).RELU for hidden layers and linear activation for the output layer.\r\n2).Yes,one-hot encoding with Feature_hashing to save memory.\r\n\r\n[/quote]\r\ncan you give me an simple example python code for solve this problem using keras?",
    "126825": "I got ~0.48 with a simple 3 layers feedforward Neural Network(1536*3),your code is right but the size of your Neural Network should be larger.",
    "129756": "How are those of you using neural networks modeling the problem? Are you simply treating all the past data as a training set and just performing a mutlinomial regression? Unless one uses some sort of sequential model, the dynamical nature of the problem will be missed; I would expect the demand of a product to only be influenced by things happening locally (a sale/promotion, public opinion, other unexplainable phenomenon).",
    "128820": "[quote=mgchbot;127083]\r\n\r\n[quote=Insaf Ashrapov;127069]\r\n\r\n@mgchbot what activation functions have you used, have you applied one-hot encoding?\r\n\r\n[/quote]\r\n\r\n1).RELU for hidden layers and linear activation for the output layer.\r\n2).Yes,one-hot encoding with Feature_hashing to save memory.\r\n\r\n[/quote]\r\n\r\n@mgchbot: what configuration do you have? Which graphic card do you use?\r\n\r\nThanks",
    "127426": "I don't know how to train such a large data set on neural network. ",
    "127106": "How many samples did you use for the training? How long did it take to train. \r\n\r\nTahnks!!",
    "127069": "@mgchbot what activation functions have you used, have you applied one-hot encoding?",
    "129176": ""
  }
}