{
  "id": 310481,
  "title": "Model giving constant loss",
  "url": "/competitions/football-match-probability-prediction/discussion/310481",
  "author_name": "",
  "post_date": "2022-03-01T15:14:29.183679500Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hey all!<br>\nI have a beginner doubt regarding this competition. So what I'm trying to do is build a simple model to predict the probabilities of away, draw, and home using a Sequential Keras Model.<br>\nI am only using all the home_team_history_rating and away_team_history_rating to get this prediction and have dropped all other categorical columns. My model is as follows - </p>\n<blockquote>\n  <blockquote>\n    <blockquote>\n      <p>model = Sequential([<br>\n          layers.Input(20),         <br>\n          layers.Dense(10),<br>\n          layers.Dense(10, activation='relu'),<br>\n          layers.Dense(3)<br>\n      ])</p>\n      <p>model.compile(optimizer='adam',<br>\n                    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),<br>\n                    metrics=['accuracy'])</p>\n    </blockquote>\n  </blockquote>\n</blockquote>\n<p>However, this model is giving me a constant loss and accuracy. Can anyone help me with this and point me in the correct direction. My notebook is public if anyone is willing to help. </p>\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "1708610",
      "postDate": "03/01/2022 15:14:29",
      "content": "<p>Hey all!<br>\nI have a beginner doubt regarding this competition. So what I'm trying to do is build a simple model to predict the probabilities of away, draw, and home using a Sequential Keras Model.<br>\nI am only using all the home_team_history_rating and away_team_history_rating to get this prediction and have dropped all other categorical columns. My model is as follows - </p>\n<blockquote>\n  <blockquote>\n    <blockquote>\n      <p>model = Sequential([<br>\n          layers.Input(20),         <br>\n          layers.Dense(10),<br>\n          layers.Dense(10, activation='relu'),<br>\n          layers.Dense(3)<br>\n      ])</p>\n      <p>model.compile(optimizer='adam',<br>\n                    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),<br>\n                    metrics=['accuracy'])</p>\n    </blockquote>\n  </blockquote>\n</blockquote>\n<p>However, this model is giving me a constant loss and accuracy. Can anyone help me with this and point me in the correct direction. My notebook is public if anyone is willing to help. </p>\n<p>Thanks!</p>",
      "rawMarkdown": "Hey all!\nI have a beginner doubt regarding this competition. So what I'm trying to do is build a simple model to predict the probabilities of away, draw, and home using a Sequential Keras Model.\nI am only using all the home_team_history_rating and away_team_history_rating to get this prediction and have dropped all other categorical columns. My model is as follows - \n\n>>>model = Sequential([\n    layers.Input(20),         \n    layers.Dense(10),\n    layers.Dense(10, activation='relu'),\n    layers.Dense(3)\n])\n\n>>>model.compile(optimizer='adam',\n              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n              metrics=['accuracy'])\n\nHowever, this model is giving me a constant loss and accuracy. Can anyone help me with this and point me in the correct direction. My notebook is public if anyone is willing to help. \n\nThanks!",
      "votes": null
    },
    {
      "id": "1708699",
      "postDate": "03/01/2022 16:25:26",
      "content": "<p>Hi, Thanks for joining the competition. Two things:</p>\n<ul>\n<li>your <code>X_train</code> contains missing values (nan). I am not even sure why your loss is not nan. Try to fill nan values with a proper number (try 0, average,  predicted missing values).</li>\n<li>Avoid the <code>relu</code> activation before the softmax, because if you have a lot of unactivated neural (0s), you are likely to predict 33% and the loss won't go down</li>\n</ul>\n<p>For instance:</p>\n<pre><code>model = Sequential([\n    layers.Input(20),\n    layers.Dense(10, activation='relu'),\n    layers.Dense(10),\n    layers.Dense(3)\n])\n\nmodel.compile(optimizer='adam',\n              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n              metrics=['accuracy'])\n\nmodel.fit(X_train.fillna(0), y_train, batch_size=1000, epochs=500, verbose=1)\n</code></pre>",
      "rawMarkdown": "Hi, Thanks for joining the competition. Two things:\n- your ```X_train``` contains missing values (nan). I am not even sure why your loss is not nan. Try to fill nan values with a proper number (try 0, average,  predicted missing values).\n- Avoid the ``relu`` activation before the softmax, because if you have a lot of unactivated neural (0s), you are likely to predict 33% and the loss won't go down\n\nFor instance:\n\n```\nmodel = Sequential([\n    layers.Input(20),\n    layers.Dense(10, activation='relu'),\n    layers.Dense(10),\n    layers.Dense(3)\n])\n\nmodel.compile(optimizer='adam',\n              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n              metrics=['accuracy'])\n\nmodel.fit(X_train.fillna(0), y_train, batch_size=1000, epochs=500, verbose=1)\n```",
      "votes": null
    },
    {
      "id": "1708707",
      "postDate": "03/01/2022 16:34:23",
      "content": "<p>Thanks for your reply! I did notice the null but didn't think that it would lead to a static model, I will definitely fix that. <br>\nThanks for the ReLu suggestion too, I did not take that into consideration!</p>",
      "rawMarkdown": "Thanks for your reply! I did notice the null but didn't think that it would lead to a static model, I will definitely fix that. \nThanks for the ReLu suggestion too, I did not take that into consideration!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1708699,
      "author_name": "octosportio",
      "author_url": "",
      "post_date": "03/01/2022 16:25:26",
      "content": "<p>Hi, Thanks for joining the competition. Two things:</p>\n<ul>\n<li>your <code>X_train</code> contains missing values (nan). I am not even sure why your loss is not nan. Try to fill nan values with a proper number (try 0, average,  predicted missing values).</li>\n<li>Avoid the <code>relu</code> activation before the softmax, because if you have a lot of unactivated neural (0s), you are likely to predict 33% and the loss won't go down</li>\n</ul>\n<p>For instance:</p>\n<pre><code>model = Sequential([\n    layers.Input(20),\n    layers.Dense(10, activation='relu'),\n    layers.Dense(10),\n    layers.Dense(3)\n])\n\nmodel.compile(optimizer='adam',\n              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n              metrics=['accuracy'])\n\nmodel.fit(X_train.fillna(0), y_train, batch_size=1000, epochs=500, verbose=1)\n</code></pre>",
      "votes": null,
      "replies": [
        {
          "id": 1708707,
          "author_name": "purvansharora",
          "author_url": "",
          "post_date": "03/01/2022 16:34:23",
          "content": "<p>Thanks for your reply! I did notice the null but didn't think that it would lead to a static model, I will definitely fix that. <br>\nThanks for the ReLu suggestion too, I did not take that into consideration!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1708610": "Hey all!\nI have a beginner doubt regarding this competition. So what I'm trying to do is build a simple model to predict the probabilities of away, draw, and home using a Sequential Keras Model.\nI am only using all the home_team_history_rating and away_team_history_rating to get this prediction and have dropped all other categorical columns. My model is as follows - \n\n>>>model = Sequential([\n    layers.Input(20),         \n    layers.Dense(10),\n    layers.Dense(10, activation='relu'),\n    layers.Dense(3)\n])\n\n>>>model.compile(optimizer='adam',\n              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n              metrics=['accuracy'])\n\nHowever, this model is giving me a constant loss and accuracy. Can anyone help me with this and point me in the correct direction. My notebook is public if anyone is willing to help. \n\nThanks!",
    "1708699": "Hi, Thanks for joining the competition. Two things:\n- your ```X_train``` contains missing values (nan). I am not even sure why your loss is not nan. Try to fill nan values with a proper number (try 0, average,  predicted missing values).\n- Avoid the ``relu`` activation before the softmax, because if you have a lot of unactivated neural (0s), you are likely to predict 33% and the loss won't go down\n\nFor instance:\n\n```\nmodel = Sequential([\n    layers.Input(20),\n    layers.Dense(10, activation='relu'),\n    layers.Dense(10),\n    layers.Dense(3)\n])\n\nmodel.compile(optimizer='adam',\n              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n              metrics=['accuracy'])\n\nmodel.fit(X_train.fillna(0), y_train, batch_size=1000, epochs=500, verbose=1)\n```",
    "1708707": "Thanks for your reply! I did notice the null but didn't think that it would lead to a static model, I will definitely fix that. \nThanks for the ReLu suggestion too, I did not take that into consideration!"
  },
  "source": "meta"
}