{
  "id": 103881,
  "title": "discriminator.trainable = False in keras !!",
  "url": "/competitions/generative-dog-images/discussion/103881",
  "author_name": "",
  "post_date": "2019-08-12T17:28:58.856837800Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>This is the following i have written for my GAN.</p>\n\n<h1>making discriminator</h1>\n\n<p>d_input = Input(shape=(2,))\nd_output = Activation('softmax')(Dense(2)(d_input))\ndiscriminator = Model(inputs=d_input, outputs=d_output)\ndiscriminator.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['categorical_accuracy'])</p>\n\n<h1>making generator</h1>\n\n<p>g_input = Input(shape=(2,))</p>\n\n<p>g_output = Activation('relu')(Dense(2)(g_input))</p>\n\n<p>generator = Model(inputs=g_input, outputs=g_output)</p>\n\n<h1>making gan(generator -&gt; discriminator)</h1>\n\n<p>discriminator.trainable = False</p>\n\n<p>gan = Model(inputs=g_input, outputs=discriminator(g_output))</p>\n\n<p>gan.compile(loss='categorical_crossentropy', optimizer='adam')</p>\n\n<p>But this is giving me this warning.\nUserWarning: Discrepancy between trainable weights and collected trainable weights, did you set <code>model.trainable</code> without calling <code>model.compile</code> after ? 'Discrepancy between trainable weights and collected trainable' when the first calling discriminator.train_on_batch</p>\n\n<p>Can anyone explain to me what is correct way to code it up?</p>",
  "messages": [
    {
      "id": "597712",
      "postDate": "08/12/2019 17:28:58",
      "content": "<p>This is the following i have written for my GAN.</p>\n\n<h1>making discriminator</h1>\n\n<p>d_input = Input(shape=(2,))\nd_output = Activation('softmax')(Dense(2)(d_input))\ndiscriminator = Model(inputs=d_input, outputs=d_output)\ndiscriminator.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['categorical_accuracy'])</p>\n\n<h1>making generator</h1>\n\n<p>g_input = Input(shape=(2,))</p>\n\n<p>g_output = Activation('relu')(Dense(2)(g_input))</p>\n\n<p>generator = Model(inputs=g_input, outputs=g_output)</p>\n\n<h1>making gan(generator -&gt; discriminator)</h1>\n\n<p>discriminator.trainable = False</p>\n\n<p>gan = Model(inputs=g_input, outputs=discriminator(g_output))</p>\n\n<p>gan.compile(loss='categorical_crossentropy', optimizer='adam')</p>\n\n<p>But this is giving me this warning.\nUserWarning: Discrepancy between trainable weights and collected trainable weights, did you set <code>model.trainable</code> without calling <code>model.compile</code> after ? 'Discrepancy between trainable weights and collected trainable' when the first calling discriminator.train_on_batch</p>\n\n<p>Can anyone explain to me what is correct way to code it up?</p>",
      "rawMarkdown": "This is the following i have written for my GAN.\n\n# making discriminator\nd_input = Input(shape=(2,))\nd_output = Activation('softmax')(Dense(2)(d_input))\ndiscriminator = Model(inputs=d_input, outputs=d_output)\ndiscriminator.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['categorical_accuracy'])\n\n# making generator\ng_input = Input(shape=(2,))\n\ng_output = Activation('relu')(Dense(2)(g_input))\n\ngenerator = Model(inputs=g_input, outputs=g_output)\n\n\n# making gan(generator -&gt; discriminator)\n\ndiscriminator.trainable = False\n\ngan = Model(inputs=g_input, outputs=discriminator(g_output))\n\ngan.compile(loss='categorical_crossentropy', optimizer='adam')\n\n\nBut this is giving me this warning.\nUserWarning: Discrepancy between trainable weights and collected trainable weights, did you set `model.trainable` without calling `model.compile` after ? 'Discrepancy between trainable weights and collected trainable' when the first calling discriminator.train_on_batch\n\nCan anyone explain to me what is correct way to code it up?",
      "votes": null
    },
    {
      "id": "598454",
      "postDate": "08/13/2019 15:35:24",
      "content": "<p>I believe you could use something like this:\n```\ndef critic(input_shape, opt=None):\n    i = L.Input(shape=input_shape)\n    x = ...  # your model's definition\n    m = models.Model(inputs=i, outputs=x, name='Critic')\n    m.compile(loss='binary_crossentropy', optimizer='adam')\n    return m</p>\n\n<p>def generator(z):\n    i = L.Input(shape=(z,))\n    x = ...  # your model's definition\n    return models.Model(inputs=i, outputs=x, name='Generator')</p>\n\n<p>def combine_as_gan(gen, cri):\n    critic.trainable = False\n    model = models.Sequential([gen, cri], name='GAN')\n    model.compile(loss='binary_crossentropy', optimizer='adam)\n    return model</p>\n\n<p>...</p>\n\n<p>g = generator(latent_dim_size)\nd = critic(input_shape)\ngan = combine_as_gan(g, d)\n```\nNot sure if any warnings were shown or not, but I was able to train a properly working GAN using this approach. Maybe there is a better way?</p>",
      "rawMarkdown": "I believe you could use something like this:\n```\ndef critic(input_shape, opt=None):\n    i = L.Input(shape=input_shape)\n    x = ...  # your model's definition\n    m = models.Model(inputs=i, outputs=x, name='Critic')\n    m.compile(loss='binary_crossentropy', optimizer='adam')\n    return m\n\ndef generator(z):\n    i = L.Input(shape=(z,))\n    x = ...  # your model's definition\n    return models.Model(inputs=i, outputs=x, name='Generator')\n\ndef combine_as_gan(gen, cri):\n    critic.trainable = False\n    model = models.Sequential([gen, cri], name='GAN')\n    model.compile(loss='binary_crossentropy', optimizer='adam)\n    return model\n\n...\n\ng = generator(latent_dim_size)\nd = critic(input_shape)\ngan = combine_as_gan(g, d)\n```\nNot sure if any warnings were shown or not, but I was able to train a properly working GAN using this approach. Maybe there is a better way?",
      "votes": null
    },
    {
      "id": "599275",
      "postDate": "08/14/2019 18:41:12",
      "content": "<p>\"<em>The warning was introduced to fix #8121 to warn users that they have to call compile again after changing .trainable. I think it make sense to have this warning because this is an otherwise surprising behaviour for new users.</em>\" </p>\n\n<p>There are several threads about this issue, i think the most informative is over \n<a href=\"https://github.com/keras-team/keras/issues/8585\">https://github.com/keras-team/keras/issues/8585</a></p>\n\n<p>Hope it helps</p>",
      "rawMarkdown": "\"*The warning was introduced to fix #8121 to warn users that they have to call compile again after changing .trainable. I think it make sense to have this warning because this is an otherwise surprising behaviour for new users.*\" \n\nThere are several threads about this issue, i think the most informative is over \nhttps://github.com/keras-team/keras/issues/8585\n\nHope it helps",
      "votes": null
    },
    {
      "id": "600073",
      "postDate": "08/15/2019 16:00:05",
      "content": "<p><a href=\"/tavoglc\">@tavoglc</a> \nWhile making gan, I had put discriminator.trainable = False</p>\n\n<p>But I had called discriminator compile before making gan</p>\n\n<p>Now my doubt is whether my discriminator will get updated or not if I train discriminator?</p>",
      "rawMarkdown": "tavoglc \nWhile making gan, I had put discriminator.trainable = False\n\nBut I had called discriminator compile before making gan\n\nNow my doubt is whether my discriminator will get updated or not if I train discriminator?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 598454,
      "author_name": "purplejester",
      "author_url": "",
      "post_date": "08/13/2019 15:35:24",
      "content": "<p>I believe you could use something like this:\n```\ndef critic(input_shape, opt=None):\n    i = L.Input(shape=input_shape)\n    x = ...  # your model's definition\n    m = models.Model(inputs=i, outputs=x, name='Critic')\n    m.compile(loss='binary_crossentropy', optimizer='adam')\n    return m</p>\n\n<p>def generator(z):\n    i = L.Input(shape=(z,))\n    x = ...  # your model's definition\n    return models.Model(inputs=i, outputs=x, name='Generator')</p>\n\n<p>def combine_as_gan(gen, cri):\n    critic.trainable = False\n    model = models.Sequential([gen, cri], name='GAN')\n    model.compile(loss='binary_crossentropy', optimizer='adam)\n    return model</p>\n\n<p>...</p>\n\n<p>g = generator(latent_dim_size)\nd = critic(input_shape)\ngan = combine_as_gan(g, d)\n```\nNot sure if any warnings were shown or not, but I was able to train a properly working GAN using this approach. Maybe there is a better way?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 599275,
      "author_name": "tavoglc",
      "author_url": "",
      "post_date": "08/14/2019 18:41:12",
      "content": "<p>\"<em>The warning was introduced to fix #8121 to warn users that they have to call compile again after changing .trainable. I think it make sense to have this warning because this is an otherwise surprising behaviour for new users.</em>\" </p>\n\n<p>There are several threads about this issue, i think the most informative is over \n<a href=\"https://github.com/keras-team/keras/issues/8585\">https://github.com/keras-team/keras/issues/8585</a></p>\n\n<p>Hope it helps</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 600073,
      "author_name": "gsh3729",
      "author_url": "",
      "post_date": "08/15/2019 16:00:05",
      "content": "<p><a href=\"/tavoglc\">@tavoglc</a> \nWhile making gan, I had put discriminator.trainable = False</p>\n\n<p>But I had called discriminator compile before making gan</p>\n\n<p>Now my doubt is whether my discriminator will get updated or not if I train discriminator?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "597712": "This is the following i have written for my GAN.\n\n# making discriminator\nd_input = Input(shape=(2,))\nd_output = Activation('softmax')(Dense(2)(d_input))\ndiscriminator = Model(inputs=d_input, outputs=d_output)\ndiscriminator.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['categorical_accuracy'])\n\n# making generator\ng_input = Input(shape=(2,))\n\ng_output = Activation('relu')(Dense(2)(g_input))\n\ngenerator = Model(inputs=g_input, outputs=g_output)\n\n\n# making gan(generator -&gt; discriminator)\n\ndiscriminator.trainable = False\n\ngan = Model(inputs=g_input, outputs=discriminator(g_output))\n\ngan.compile(loss='categorical_crossentropy', optimizer='adam')\n\n\nBut this is giving me this warning.\nUserWarning: Discrepancy between trainable weights and collected trainable weights, did you set `model.trainable` without calling `model.compile` after ? 'Discrepancy between trainable weights and collected trainable' when the first calling discriminator.train_on_batch\n\nCan anyone explain to me what is correct way to code it up?",
    "598454": "I believe you could use something like this:\n```\ndef critic(input_shape, opt=None):\n    i = L.Input(shape=input_shape)\n    x = ...  # your model's definition\n    m = models.Model(inputs=i, outputs=x, name='Critic')\n    m.compile(loss='binary_crossentropy', optimizer='adam')\n    return m\n\ndef generator(z):\n    i = L.Input(shape=(z,))\n    x = ...  # your model's definition\n    return models.Model(inputs=i, outputs=x, name='Generator')\n\ndef combine_as_gan(gen, cri):\n    critic.trainable = False\n    model = models.Sequential([gen, cri], name='GAN')\n    model.compile(loss='binary_crossentropy', optimizer='adam)\n    return model\n\n...\n\ng = generator(latent_dim_size)\nd = critic(input_shape)\ngan = combine_as_gan(g, d)\n```\nNot sure if any warnings were shown or not, but I was able to train a properly working GAN using this approach. Maybe there is a better way?",
    "599275": "\"*The warning was introduced to fix #8121 to warn users that they have to call compile again after changing .trainable. I think it make sense to have this warning because this is an otherwise surprising behaviour for new users.*\" \n\nThere are several threads about this issue, i think the most informative is over \nhttps://github.com/keras-team/keras/issues/8585\n\nHope it helps",
    "600073": "tavoglc \nWhile making gan, I had put discriminator.trainable = False\n\nBut I had called discriminator compile before making gan\n\nNow my doubt is whether my discriminator will get updated or not if I train discriminator?"
  },
  "source": "meta"
}