{
  "id": 186853,
  "title": "Creating a Instance normalization layer in tensorflow",
  "url": "/competitions/gan-getting-started/discussion/186853",
  "author_name": "",
  "post_date": "2020-09-26T09:53:50.547718100Z",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Instance normalization is similar to other layer normalization techniques, it computes the mean/standard deviation and normalizes across each channel in each training example.</p>\n<p>It replaces batch normalization from many generative tasks. It was first devised for fast style transfer and is now used with GANs too and provides much appealing results.</p>\n<pre><code>class InstanceNormalization(tf.keras.layers.Layer):\n    def __init__(self,**kwargs):\n        super(InstanceNormalization, self).__init__(**kwargs)\n    def call(self,inputs):\n        batch, rows, cols, channels = [i for i in inputs.get_shape()]\n        mu, var = tf.nn.moments(inputs, [1,2], keepdims=True)\n        shift = tf.Variable(tf.zeros([channels]))\n        scale = tf.Variable(tf.ones([channels]))\n        epsilon = 1e-3\n        normalized = (inputs-mu)/tf.sqrt(var + epsilon)\n        return scale * normalized + shift\n</code></pre>",
  "messages": [
    {
      "id": "1027693",
      "postDate": "09/26/2020 09:53:50",
      "content": "<p>Instance normalization is similar to other layer normalization techniques, it computes the mean/standard deviation and normalizes across each channel in each training example.</p>\n<p>It replaces batch normalization from many generative tasks. It was first devised for fast style transfer and is now used with GANs too and provides much appealing results.</p>\n<pre><code>class InstanceNormalization(tf.keras.layers.Layer):\n    def __init__(self,**kwargs):\n        super(InstanceNormalization, self).__init__(**kwargs)\n    def call(self,inputs):\n        batch, rows, cols, channels = [i for i in inputs.get_shape()]\n        mu, var = tf.nn.moments(inputs, [1,2], keepdims=True)\n        shift = tf.Variable(tf.zeros([channels]))\n        scale = tf.Variable(tf.ones([channels]))\n        epsilon = 1e-3\n        normalized = (inputs-mu)/tf.sqrt(var + epsilon)\n        return scale * normalized + shift\n</code></pre>",
      "rawMarkdown": "Instance normalization is similar to other layer normalization techniques, it computes the mean/standard deviation and normalizes across each channel in each training example.\n\nIt replaces batch normalization from many generative tasks. It was first devised for fast style transfer and is now used with GANs too and provides much appealing results.\n\n```python\nclass InstanceNormalization(tf.keras.layers.Layer):\n    def __init__(self,**kwargs):\n        super(InstanceNormalization, self).__init__(**kwargs)\n    def call(self,inputs):\n        batch, rows, cols, channels = [i for i in inputs.get_shape()]\n        mu, var = tf.nn.moments(inputs, [1,2], keepdims=True)\n        shift = tf.Variable(tf.zeros([channels]))\n        scale = tf.Variable(tf.ones([channels]))\n        epsilon = 1e-3\n        normalized = (inputs-mu)/tf.sqrt(var + epsilon)\n        return scale * normalized + shift\n```",
      "votes": null
    },
    {
      "id": "1078288",
      "postDate": "11/14/2020 15:43:54",
      "content": "<p>This didn't work in my code (tensorflow 2.3).<br>\nIt seems that Variable is not available in the <code>__call__</code>, so I modified it and used it as follows</p>\n<pre><code>class InstanceNormalization(tf.keras.layers.Layer):\n\n    def __init__(self, channels, **kwargs):\n        super(InstanceNormalization, self).__init__(**kwargs)\n        self.epsilon = 1e-3\n        self.shift = tf.Variable(tf.zeros([channels]))\n        self.scale = tf.Variable(tf.ones([channels]))\n\n    def call(self, inputs):\n        mu, var = tf.nn.moments(inputs, [1,2], keepdims=True)\n        normalized = (inputs-mu) / tf.sqrt(var + self.epsilon)\n        return self.scale * normalized + self.shift\n</code></pre>\n<p>Thanks.</p>",
      "rawMarkdown": "This didn't work in my code (tensorflow 2.3).\nIt seems that Variable is not available in the `__call__`, so I modified it and used it as follows\n\n```\nclass InstanceNormalization(tf.keras.layers.Layer):\n    \n    def __init__(self, channels, **kwargs):\n        super(InstanceNormalization, self).__init__(**kwargs)\n        self.epsilon = 1e-3\n        self.shift = tf.Variable(tf.zeros([channels]))\n        self.scale = tf.Variable(tf.ones([channels]))\n        \n    def call(self, inputs):\n        mu, var = tf.nn.moments(inputs, [1,2], keepdims=True)\n        normalized = (inputs-mu) / tf.sqrt(var + self.epsilon)\n        return self.scale * normalized + self.shift\n```\n\nThanks.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1078288,
      "author_name": "stealthflow",
      "author_url": "",
      "post_date": "11/14/2020 15:43:54",
      "content": "<p>This didn't work in my code (tensorflow 2.3).<br>\nIt seems that Variable is not available in the <code>__call__</code>, so I modified it and used it as follows</p>\n<pre><code>class InstanceNormalization(tf.keras.layers.Layer):\n\n    def __init__(self, channels, **kwargs):\n        super(InstanceNormalization, self).__init__(**kwargs)\n        self.epsilon = 1e-3\n        self.shift = tf.Variable(tf.zeros([channels]))\n        self.scale = tf.Variable(tf.ones([channels]))\n\n    def call(self, inputs):\n        mu, var = tf.nn.moments(inputs, [1,2], keepdims=True)\n        normalized = (inputs-mu) / tf.sqrt(var + self.epsilon)\n        return self.scale * normalized + self.shift\n</code></pre>\n<p>Thanks.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1027693": "Instance normalization is similar to other layer normalization techniques, it computes the mean/standard deviation and normalizes across each channel in each training example.\n\nIt replaces batch normalization from many generative tasks. It was first devised for fast style transfer and is now used with GANs too and provides much appealing results.\n\n```python\nclass InstanceNormalization(tf.keras.layers.Layer):\n    def __init__(self,**kwargs):\n        super(InstanceNormalization, self).__init__(**kwargs)\n    def call(self,inputs):\n        batch, rows, cols, channels = [i for i in inputs.get_shape()]\n        mu, var = tf.nn.moments(inputs, [1,2], keepdims=True)\n        shift = tf.Variable(tf.zeros([channels]))\n        scale = tf.Variable(tf.ones([channels]))\n        epsilon = 1e-3\n        normalized = (inputs-mu)/tf.sqrt(var + epsilon)\n        return scale * normalized + shift\n```",
    "1078288": "This didn't work in my code (tensorflow 2.3).\nIt seems that Variable is not available in the `__call__`, so I modified it and used it as follows\n\n```\nclass InstanceNormalization(tf.keras.layers.Layer):\n    \n    def __init__(self, channels, **kwargs):\n        super(InstanceNormalization, self).__init__(**kwargs)\n        self.epsilon = 1e-3\n        self.shift = tf.Variable(tf.zeros([channels]))\n        self.scale = tf.Variable(tf.ones([channels]))\n        \n    def call(self, inputs):\n        mu, var = tf.nn.moments(inputs, [1,2], keepdims=True)\n        normalized = (inputs-mu) / tf.sqrt(var + self.epsilon)\n        return self.scale * normalized + self.shift\n```\n\nThanks."
  },
  "source": "meta"
}