{
  "id": 182536,
  "title": "EfficientDet Explained",
  "url": "/competitions/global-wheat-detection/discussion/182536",
  "author_name": "",
  "post_date": "2020-09-13T09:29:13.816362700Z",
  "votes": 27,
  "comment_count": 12,
  "views": 0,
  "content": "<p>What is <strong>EfficientDet</strong> and how does it work?</p>\n<p>In short, it is a recent (first submitted at the end of 2019, accepted in CVPR in 2020) very efficient (surprise surprise) object detection model.</p>\n<p>In more details, it is a family of models designed by researchers from <strong>Google brain</strong>. The interesting thing is that some parts of it were designed using automatic <strong>architecture search</strong>, i.e. there was a meta-model that was trained to find the best hyper-parameters of the trained model automatically. Let's dive in more details.</p>\n<p>First, let's start with the model's <strong>architecture</strong>: </p>\n<p><img src=\"https://camo.githubusercontent.com/0ebd0147a2340c5ef6c3e30f2e51ab0cc812e515/68747470733a2f2f73756e392d33352e757365726170692e636f6d2f633230353632382f763230353632383732362f64323962342f67546a705534676a327a632e6a7067\"></p>\n<p>As you can see, there are <strong>3 main</strong> building blocks:</p>\n<ol>\n<li>a <strong>classification</strong> backbone block</li>\n<li>a <strong>multi-scale features fusion</strong> block</li>\n<li>a <strong>class</strong> and <strong>bounding box</strong> networks block</li>\n</ol>\n<p>In more details, we have: </p>\n<ul>\n<li>an <strong>EfficientNet</strong> (<strong>Net</strong> and not <strong>Det</strong>) backbone. This backbone was found using NAS (neural-architecture search) through the use of the MnasNet framework. More on this in the next section</li>\n<li><strong>BiFPN</strong> as a multi-scale fusion layer: once we have a classification network we need to efficiently extract features for the detection step. One extra challenge is to do so for multiple scales (learn more about this <a href=\"https://d2l.ai/chapter_computer-vision/multiscale-object-detection.html\" target=\"_blank\">here</a>). These two challenges are solved efficiently using the BiFPN layer. Again, more on this in the next section.</li>\n<li><strong>Two heads</strong>: finally, everything is connected to two output networks. One for predicting the most likely label and one for predicting the detected bounding box (if any).</li>\n</ul>\n<p>To experiment with the model and learn more, check this <a href=\"https://github.com/google/automl/blob/master/efficientdet/tutorial.ipynb\" target=\"_blank\">colab notebook</a>, the original <a href=\"https://arxiv.org/pdf/1911.09070.pdf\" target=\"_blank\">paper</a>, and the original <a href=\"https://github.com/google/automl/tree/master/efficientdet\" target=\"_blank\">implementation</a>. </p>\n<p>In the following sections, we will explore each part in more details. Let's go!</p>\n<h2>EfficientNet as a Backbone</h2>\n<p>This is the first part of the architecture. It uses the <strong>EfficientNet</strong> architecture and includes pre-trained weights.</p>\n<p>Here is a look at the overall architecture:</p>\n<p><img src=\"https://miro.medium.com/max/1395/1*O1TZKZjafr9qqkxHxkJmAQ.png\"></p>\n<p>As you can see, the interesting thing is that many elements of the model have been optimized using neural automatic search: <strong>width</strong>, <strong>depth</strong>, <strong>number of channels</strong>, and <strong>resolution</strong>.</p>\n<p>To learn more about this <strong>EfficientNet</strong> backbone, read the original paper <a href=\"https://arxiv.org/pdf/1905.11946.pdf\" target=\"_blank\">here</a>, the blog <a href=\"https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html\" target=\"_blank\">post</a>, and the implementation <a href=\"https://github.com/google/automl/tree/master/efficientdet/backbone\" target=\"_blank\">here</a> (from the <strong>efficientdet</strong> repo).</p>\n<h2>BiFPN as a Multi-Scale Fusion Layer</h2>\n<p>Once we have a classification network, one important question to ask is: how to move from the <strong>classification</strong> model to an <strong>object detection</strong> one? This is cleverly solved by using features maps (i.e. intermediate learned representations). Another question is how to detect multiple objects having different sizes (for example a small cat and a car on the same image)? This is solved by taking features maps having different sizes. </p>\n<p>In most modern object detection architectures, both problems are solved at once using an FPN layer. BiFPN, as its name indicates, builds upon the ideas of the FPN layer and adds one simple idea: instead of simply aggregating the different representations in a top-down fashion, it takes the PANet approach (i.e. top-down and bottom-up connections) and optimizes cross-scale connections.</p>\n<p>One last trick is to use a weighted feature fusion instead of the unweighted FPN approach (the weights are learned as well) and normalize these weights using a fast normalization procedure (i.e. normalize by the sum instead of using softmax). Here is one example of a BiFPN computation from the paper:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F612b65f04d5732669dbe04d5f0fc907d%2Fbifpn_computation_example.png?generation=1601114628541834&amp;alt=media\"></p>\n<p>All these tricks make the BiFPN layer both efficient and accurate. We will see one additional trick applied in the next and last section. Check the following graph to learn more about other FPN variations (including PANet and BiFPN):</p>\n<p><img src=\"https://miro.medium.com/max/840/1*funGGmYqT6f2VkRnhqd2rA.png\"></p>\n<p>Finally, to learn more about FPN and BiFPN, check the paperswithcode <a href=\"https://paperswithcode.com/method/bifpn\" target=\"_blank\">page</a>, this <a href=\"https://towardsdatascience.com/review-fpn-feature-pyramid-network-object-detection-262fc7482610\" target=\"_blank\">blog post</a>, and this <a href=\"https://medium.com/@jonathan_hui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106c\" target=\"_blank\">one</a> as well. </p>\n<p>Let's move to the next step: the prediction networks.</p>\n<h2>Two Heads</h2>\n<p>The third ingredient to the mix is the <strong>two heads network</strong>:</p>\n<ul>\n<li>one for predicting the coordinates of the <strong>bounding box</strong></li>\n<li>one for predicting the <strong>class</strong> (the probability of each class to be more precise)</li>\n</ul>\n<p>Each of the two networks take as input all the outputs of the previous <strong>BiFPN layer</strong>. Then we have two fully-connected layers before getting the final values. As simple as that!</p>\n<p>We close this section with the last trick: compound scaling.</p>\n<h2>Compound Scaling</h2>\n<p>One last ingredient to the mix is how the network is scaled. Indeed, this is a trick often used to improve the performance <br>\nof networks. The most common way to do it is to scale the backbone's width and depth. </p>\n<p>In the EfficientDet model, the EffincientNet backbone is scaled reusing the same values in the original network.<br>\nThe interesting additional thing is the scaling of the BiFPN and prediction networks.<br>\nIn fact, instead of using only one BiFPN and one prediction networks, these components are repeated. Finally, width (W), depth (D), and input resolution (R) in the different sections are scaled using one single parameter $\\phi$ in this fashion:</p>\n<ul>\n<li>1. $$W_{bifpn} = 64 . (1.35^{\\phi})$$ and $$D_{bifpn} = 3 + \\phi$$ for the <strong>BiFPN</strong> part</li>\n<li>2. $$W_{pred} = W_{bifpn}$$ and and $$D_{bifpn} = 3 + \\phi$$ for the <strong>prediction</strong> part</li>\n<li>3. $$R_{input} = 512 + \\phi . 128$$ for the input image resolution. Notice that it should be divisible by 128 since we take features from levels 3 to 7: $$128 = 2 ^{7}$$</li>\n</ul>\n<p>Here is the table for the different <strong>EfficientDet</strong> variants: the bigger phi is, the bigger the model:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F4b349ddd45bddd490ae3bc5079cc0e25%2Fefficientdet_scalings.png?generation=1601122425468830&amp;alt=media\"></p>\n<p>To finish this section, notice that the <strong>1.35</strong> value in the first equation has been optimized using grid search (i.e. different values are tested and the one giving the best score is selected) over [1.2, 1.25, 1.3, 1.35, 1.4, 1.45] values.</p>\n<p>That's it for the model's architecture. Time to move to the application!</p>\n<p>That's it! I hope this (long) explanation was useful. Let me know in the comments if I have missed something important. <br>\nThanks.</p>",
  "messages": [
    {
      "id": "1008634",
      "postDate": "09/13/2020 09:29:13",
      "content": "<p>What is <strong>EfficientDet</strong> and how does it work?</p>\n<p>In short, it is a recent (first submitted at the end of 2019, accepted in CVPR in 2020) very efficient (surprise surprise) object detection model.</p>\n<p>In more details, it is a family of models designed by researchers from <strong>Google brain</strong>. The interesting thing is that some parts of it were designed using automatic <strong>architecture search</strong>, i.e. there was a meta-model that was trained to find the best hyper-parameters of the trained model automatically. Let's dive in more details.</p>\n<p>First, let's start with the model's <strong>architecture</strong>: </p>\n<p><img src=\"https://camo.githubusercontent.com/0ebd0147a2340c5ef6c3e30f2e51ab0cc812e515/68747470733a2f2f73756e392d33352e757365726170692e636f6d2f633230353632382f763230353632383732362f64323962342f67546a705534676a327a632e6a7067\"></p>\n<p>As you can see, there are <strong>3 main</strong> building blocks:</p>\n<ol>\n<li>a <strong>classification</strong> backbone block</li>\n<li>a <strong>multi-scale features fusion</strong> block</li>\n<li>a <strong>class</strong> and <strong>bounding box</strong> networks block</li>\n</ol>\n<p>In more details, we have: </p>\n<ul>\n<li>an <strong>EfficientNet</strong> (<strong>Net</strong> and not <strong>Det</strong>) backbone. This backbone was found using NAS (neural-architecture search) through the use of the MnasNet framework. More on this in the next section</li>\n<li><strong>BiFPN</strong> as a multi-scale fusion layer: once we have a classification network we need to efficiently extract features for the detection step. One extra challenge is to do so for multiple scales (learn more about this <a href=\"https://d2l.ai/chapter_computer-vision/multiscale-object-detection.html\" target=\"_blank\">here</a>). These two challenges are solved efficiently using the BiFPN layer. Again, more on this in the next section.</li>\n<li><strong>Two heads</strong>: finally, everything is connected to two output networks. One for predicting the most likely label and one for predicting the detected bounding box (if any).</li>\n</ul>\n<p>To experiment with the model and learn more, check this <a href=\"https://github.com/google/automl/blob/master/efficientdet/tutorial.ipynb\" target=\"_blank\">colab notebook</a>, the original <a href=\"https://arxiv.org/pdf/1911.09070.pdf\" target=\"_blank\">paper</a>, and the original <a href=\"https://github.com/google/automl/tree/master/efficientdet\" target=\"_blank\">implementation</a>. </p>\n<p>In the following sections, we will explore each part in more details. Let's go!</p>\n<h2>EfficientNet as a Backbone</h2>\n<p>This is the first part of the architecture. It uses the <strong>EfficientNet</strong> architecture and includes pre-trained weights.</p>\n<p>Here is a look at the overall architecture:</p>\n<p><img src=\"https://miro.medium.com/max/1395/1*O1TZKZjafr9qqkxHxkJmAQ.png\"></p>\n<p>As you can see, the interesting thing is that many elements of the model have been optimized using neural automatic search: <strong>width</strong>, <strong>depth</strong>, <strong>number of channels</strong>, and <strong>resolution</strong>.</p>\n<p>To learn more about this <strong>EfficientNet</strong> backbone, read the original paper <a href=\"https://arxiv.org/pdf/1905.11946.pdf\" target=\"_blank\">here</a>, the blog <a href=\"https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html\" target=\"_blank\">post</a>, and the implementation <a href=\"https://github.com/google/automl/tree/master/efficientdet/backbone\" target=\"_blank\">here</a> (from the <strong>efficientdet</strong> repo).</p>\n<h2>BiFPN as a Multi-Scale Fusion Layer</h2>\n<p>Once we have a classification network, one important question to ask is: how to move from the <strong>classification</strong> model to an <strong>object detection</strong> one? This is cleverly solved by using features maps (i.e. intermediate learned representations). Another question is how to detect multiple objects having different sizes (for example a small cat and a car on the same image)? This is solved by taking features maps having different sizes. </p>\n<p>In most modern object detection architectures, both problems are solved at once using an FPN layer. BiFPN, as its name indicates, builds upon the ideas of the FPN layer and adds one simple idea: instead of simply aggregating the different representations in a top-down fashion, it takes the PANet approach (i.e. top-down and bottom-up connections) and optimizes cross-scale connections.</p>\n<p>One last trick is to use a weighted feature fusion instead of the unweighted FPN approach (the weights are learned as well) and normalize these weights using a fast normalization procedure (i.e. normalize by the sum instead of using softmax). Here is one example of a BiFPN computation from the paper:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F612b65f04d5732669dbe04d5f0fc907d%2Fbifpn_computation_example.png?generation=1601114628541834&amp;alt=media\"></p>\n<p>All these tricks make the BiFPN layer both efficient and accurate. We will see one additional trick applied in the next and last section. Check the following graph to learn more about other FPN variations (including PANet and BiFPN):</p>\n<p><img src=\"https://miro.medium.com/max/840/1*funGGmYqT6f2VkRnhqd2rA.png\"></p>\n<p>Finally, to learn more about FPN and BiFPN, check the paperswithcode <a href=\"https://paperswithcode.com/method/bifpn\" target=\"_blank\">page</a>, this <a href=\"https://towardsdatascience.com/review-fpn-feature-pyramid-network-object-detection-262fc7482610\" target=\"_blank\">blog post</a>, and this <a href=\"https://medium.com/@jonathan_hui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106c\" target=\"_blank\">one</a> as well. </p>\n<p>Let's move to the next step: the prediction networks.</p>\n<h2>Two Heads</h2>\n<p>The third ingredient to the mix is the <strong>two heads network</strong>:</p>\n<ul>\n<li>one for predicting the coordinates of the <strong>bounding box</strong></li>\n<li>one for predicting the <strong>class</strong> (the probability of each class to be more precise)</li>\n</ul>\n<p>Each of the two networks take as input all the outputs of the previous <strong>BiFPN layer</strong>. Then we have two fully-connected layers before getting the final values. As simple as that!</p>\n<p>We close this section with the last trick: compound scaling.</p>\n<h2>Compound Scaling</h2>\n<p>One last ingredient to the mix is how the network is scaled. Indeed, this is a trick often used to improve the performance <br>\nof networks. The most common way to do it is to scale the backbone's width and depth. </p>\n<p>In the EfficientDet model, the EffincientNet backbone is scaled reusing the same values in the original network.<br>\nThe interesting additional thing is the scaling of the BiFPN and prediction networks.<br>\nIn fact, instead of using only one BiFPN and one prediction networks, these components are repeated. Finally, width (W), depth (D), and input resolution (R) in the different sections are scaled using one single parameter $\\phi$ in this fashion:</p>\n<ul>\n<li>1. $$W_{bifpn} = 64 . (1.35^{\\phi})$$ and $$D_{bifpn} = 3 + \\phi$$ for the <strong>BiFPN</strong> part</li>\n<li>2. $$W_{pred} = W_{bifpn}$$ and and $$D_{bifpn} = 3 + \\phi$$ for the <strong>prediction</strong> part</li>\n<li>3. $$R_{input} = 512 + \\phi . 128$$ for the input image resolution. Notice that it should be divisible by 128 since we take features from levels 3 to 7: $$128 = 2 ^{7}$$</li>\n</ul>\n<p>Here is the table for the different <strong>EfficientDet</strong> variants: the bigger phi is, the bigger the model:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F4b349ddd45bddd490ae3bc5079cc0e25%2Fefficientdet_scalings.png?generation=1601122425468830&amp;alt=media\"></p>\n<p>To finish this section, notice that the <strong>1.35</strong> value in the first equation has been optimized using grid search (i.e. different values are tested and the one giving the best score is selected) over [1.2, 1.25, 1.3, 1.35, 1.4, 1.45] values.</p>\n<p>That's it for the model's architecture. Time to move to the application!</p>\n<p>That's it! I hope this (long) explanation was useful. Let me know in the comments if I have missed something important. <br>\nThanks.</p>",
      "rawMarkdown": "What is **EfficientDet** and how does it work?\n\n\nIn short, it is a recent (first submitted at the end of 2019, accepted in CVPR in 2020) very efficient (surprise surprise) object detection model.\n\nIn more details, it is a family of models designed by researchers from **Google brain**. The interesting thing is that some parts of it were designed using automatic **architecture search**, i.e. there was a meta-model that was trained to find the best hyper-parameters of the trained model automatically. Let's dive in more details.\n\n\n\nFirst, let's start with the model's **architecture**: \n\n<img src=\"https://camo.githubusercontent.com/0ebd0147a2340c5ef6c3e30f2e51ab0cc812e515/68747470733a2f2f73756e392d33352e757365726170692e636f6d2f633230353632382f763230353632383732362f64323962342f67546a705534676a327a632e6a7067\" width=480>\n\n\nAs you can see, there are **3 main** building blocks:\n\n1. a **classification** backbone block\n2. a **multi-scale features fusion** block\n3. a **class** and **bounding box** networks block\n\nIn more details, we have: \n\n* an **EfficientNet** (**Net** and not **Det**) backbone. This backbone was found using NAS (neural-architecture search) through the use of the MnasNet framework. More on this in the next section\n* **BiFPN** as a multi-scale fusion layer: once we have a classification network we need to efficiently extract features for the detection step. One extra challenge is to do so for multiple scales (learn more about this [here](https://d2l.ai/chapter_computer-vision/multiscale-object-detection.html)). These two challenges are solved efficiently using the BiFPN layer. Again, more on this in the next section.\n* **Two heads**: finally, everything is connected to two output networks. One for predicting the most likely label and one for predicting the detected bounding box (if any).\n\n\nTo experiment with the model and learn more, check this [colab notebook](https://github.com/google/automl/blob/master/efficientdet/tutorial.ipynb), the original [paper](https://arxiv.org/pdf/1911.09070.pdf), and the original [implementation](https://github.com/google/automl/tree/master/efficientdet). \n\n\nIn the following sections, we will explore each part in more details. Let's go!\n\n## EfficientNet as a Backbone\n\nThis is the first part of the architecture. It uses the **EfficientNet** architecture and includes pre-trained weights.\n\nHere is a look at the overall architecture:\n\n\n\n<img src=\"https://miro.medium.com/max/1395/1*O1TZKZjafr9qqkxHxkJmAQ.png\" width=480>\n\nAs you can see, the interesting thing is that many elements of the model have been optimized using neural automatic search: **width**, **depth**, **number of channels**, and **resolution**.\n\nTo learn more about this **EfficientNet** backbone, read the original paper [here](https://arxiv.org/pdf/1905.11946.pdf), the blog [post](https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html), and the implementation [here](https://github.com/google/automl/tree/master/efficientdet/backbone) (from the **efficientdet** repo).\n\n\n## BiFPN as a Multi-Scale Fusion Layer\n\nOnce we have a classification network, one important question to ask is: how to move from the **classification** model to an **object detection** one? This is cleverly solved by using features maps (i.e. intermediate learned representations). Another question is how to detect multiple objects having different sizes (for example a small cat and a car on the same image)? This is solved by taking features maps having different sizes. \n\nIn most modern object detection architectures, both problems are solved at once using an FPN layer. BiFPN, as its name indicates, builds upon the ideas of the FPN layer and adds one simple idea: instead of simply aggregating the different representations in a top-down fashion, it takes the PANet approach (i.e. top-down and bottom-up connections) and optimizes cross-scale connections.\n\nOne last trick is to use a weighted feature fusion instead of the unweighted FPN approach (the weights are learned as well) and normalize these weights using a fast normalization procedure (i.e. normalize by the sum instead of using softmax). Here is one example of a BiFPN computation from the paper:\n\n\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F612b65f04d5732669dbe04d5f0fc907d%2Fbifpn_computation_example.png?generation=1601114628541834&alt=media\" width=480>\n\n\nAll these tricks make the BiFPN layer both efficient and accurate. We will see one additional trick applied in the next and last section. Check the following graph to learn more about other FPN variations (including PANet and BiFPN):\n\n\n<img src=\"https://miro.medium.com/max/840/1*funGGmYqT6f2VkRnhqd2rA.png\" ![width=480](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F1ebba71862ecd6548995a9be7e11b9dd%2Fbifpn_computation_example.png?generation=1601114578505496&alt=media\" width=480>\n\n\n\nFinally, to learn more about FPN and BiFPN, check the paperswithcode [page](https://paperswithcode.com/method/bifpn), this [blog post](https://towardsdatascience.com/review-fpn-feature-pyramid-network-object-detection-262fc7482610), and this [one](https://medium.com/@jonathan_hui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106c) as well. \n\nLet's move to the next step: the prediction networks.\n\n\n## Two Heads\n\nThe third ingredient to the mix is the **two heads network**:\n\n- one for predicting the coordinates of the **bounding box**\n- one for predicting the **class** (the probability of each class to be more precise)\n\nEach of the two networks take as input all the outputs of the previous **BiFPN layer**. Then we have two fully-connected layers before getting the final values. As simple as that!\n\nWe close this section with the last trick: compound scaling.\n\n\n## Compound Scaling\n\n\nOne last ingredient to the mix is how the network is scaled. Indeed, this is a trick often used to improve the performance \nof networks. The most common way to do it is to scale the backbone's width and depth. \n\nIn the EfficientDet model, the EffincientNet backbone is scaled reusing the same values in the original network.\nThe interesting additional thing is the scaling of the BiFPN and prediction networks.\nIn fact, instead of using only one BiFPN and one prediction networks, these components are repeated. Finally, width (W), depth (D), and input resolution (R) in the different sections are scaled using one single parameter $\\phi$ in this fashion:\n\n\n\n- 1. $$W_{bifpn} = 64 . (1.35^{\\phi})$$ and $$D_{bifpn} = 3 + \\phi$$ for the **BiFPN** part\n- 2. $$W_{pred} = W_{bifpn}$$ and and $$D_{bifpn} = 3 + \\phi$$ for the **prediction** part\n- 3. $$R_{input} = 512 + \\phi . 128$$ for the input image resolution. Notice that it should be divisible by 128 since we take features from levels 3 to 7: $$128 = 2 ^{7}$$\n\n\nHere is the table for the different **EfficientDet** variants: the bigger phi is, the bigger the model:\n    \n    \n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F4b349ddd45bddd490ae3bc5079cc0e25%2Fefficientdet_scalings.png?generation=1601122425468830&alt=media\" width=480>\n\n\nTo finish this section, notice that the **1.35** value in the first equation has been optimized using grid search (i.e. different values are tested and the one giving the best score is selected) over [1.2, 1.25, 1.3, 1.35, 1.4, 1.45] values.\n\n\nThat's it for the model's architecture. Time to move to the application!\n\n\nThat's it! I hope this (long) explanation was useful. Let me know in the comments if I have missed something important. \nThanks.",
      "votes": null
    },
    {
      "id": "1008749",
      "postDate": "09/13/2020 11:02:06",
      "content": "<p>Loved it !!<br>\nThanks for sharing !!</p>",
      "rawMarkdown": "Loved it !!\nThanks for sharing !!",
      "votes": null
    },
    {
      "id": "1008762",
      "postDate": "09/13/2020 11:16:26",
      "content": "<p>Glad it helps. For your information, this isn't yet 100% finished so stay tuned for more. </p>",
      "rawMarkdown": "Glad it helps. For your information, this isn't yet 100% finished so stay tuned for more.",
      "votes": null
    },
    {
      "id": "1008787",
      "postDate": "09/13/2020 11:56:58",
      "content": "<p>Yeah, would like to learn more from you.</p>",
      "rawMarkdown": "Yeah, would like to learn more from you.",
      "votes": null
    },
    {
      "id": "1021556",
      "postDate": "09/22/2020 01:47:59",
      "content": "<p>thanks for sharing. I'm newbie. It's very useful</p>",
      "rawMarkdown": "thanks for sharing. I'm newbie. It's very useful",
      "votes": null
    },
    {
      "id": "1027756",
      "postDate": "09/26/2020 10:34:11",
      "content": "<p>I am glad it helps! Check it again, I have improved some sections and added missing ones. </p>",
      "rawMarkdown": "I am glad it helps! Check it again, I have improved some sections and added missing ones.",
      "votes": null
    },
    {
      "id": "1027841",
      "postDate": "09/26/2020 12:22:26",
      "content": "<p>[EDIT 26-9-2020] Finished the remaining sections (BiFPN and compound scaling) and cleaned up few things. </p>",
      "rawMarkdown": "[EDIT 26-9-2020] Finished the remaining sections (BiFPN and compound scaling) and cleaned up few things.",
      "votes": null
    },
    {
      "id": "1027898",
      "postDate": "09/26/2020 13:09:53",
      "content": "<p>Good work ! keep it up </p>",
      "rawMarkdown": "Good work ! keep it up",
      "votes": null
    },
    {
      "id": "1027935",
      "postDate": "09/26/2020 13:33:10",
      "content": "<p>Cheers! I hope it was of some use. ;)</p>",
      "rawMarkdown": "Cheers! I hope it was of some use. ;)",
      "votes": null
    },
    {
      "id": "1027943",
      "postDate": "09/26/2020 13:38:49",
      "content": "<p>yes<br>\ngo ahead :D</p>",
      "rawMarkdown": "yes\ngo ahead :D",
      "votes": null
    },
    {
      "id": "1027965",
      "postDate": "09/26/2020 13:42:54",
      "content": "<p>Good job, continue like this ! BRAVO </p>",
      "rawMarkdown": "Good job, continue like this ! BRAVO",
      "votes": null
    },
    {
      "id": "1028564",
      "postDate": "09/27/2020 01:37:32",
      "content": "<p>Good Job! Still learning.</p>",
      "rawMarkdown": "Good Job! Still learning.",
      "votes": null
    },
    {
      "id": "1028799",
      "postDate": "09/27/2020 08:07:32",
      "content": "<p>Keep learning! </p>",
      "rawMarkdown": "Keep learning!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1008749,
      "author_name": "oneplustricks",
      "author_url": "",
      "post_date": "09/13/2020 11:02:06",
      "content": "<p>Loved it !!<br>\nThanks for sharing !!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1008762,
          "author_name": "yassinealouini",
          "author_url": "",
          "post_date": "09/13/2020 11:16:26",
          "content": "<p>Glad it helps. For your information, this isn't yet 100% finished so stay tuned for more. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1008787,
          "author_name": "oneplustricks",
          "author_url": "",
          "post_date": "09/13/2020 11:56:58",
          "content": "<p>Yeah, would like to learn more from you.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1021556,
      "author_name": "ttcong194",
      "author_url": "",
      "post_date": "09/22/2020 01:47:59",
      "content": "<p>thanks for sharing. I'm newbie. It's very useful</p>",
      "votes": null,
      "replies": [
        {
          "id": 1027756,
          "author_name": "yassinealouini",
          "author_url": "",
          "post_date": "09/26/2020 10:34:11",
          "content": "<p>I am glad it helps! Check it again, I have improved some sections and added missing ones. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1027841,
      "author_name": "yassinealouini",
      "author_url": "",
      "post_date": "09/26/2020 12:22:26",
      "content": "<p>[EDIT 26-9-2020] Finished the remaining sections (BiFPN and compound scaling) and cleaned up few things. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1028564,
      "author_name": "doubleliu",
      "author_url": "",
      "post_date": "09/27/2020 01:37:32",
      "content": "<p>Good Job! Still learning.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1028799,
          "author_name": "yassinealouini",
          "author_url": "",
          "post_date": "09/27/2020 08:07:32",
          "content": "<p>Keep learning! </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1027898,
      "author_name": "naim99",
      "author_url": "",
      "post_date": "09/26/2020 13:09:53",
      "content": "<p>Good work ! keep it up </p>",
      "votes": null,
      "replies": [
        {
          "id": 1027935,
          "author_name": "yassinealouini",
          "author_url": "",
          "post_date": "09/26/2020 13:33:10",
          "content": "<p>Cheers! I hope it was of some use. ;)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1027943,
          "author_name": "naim99",
          "author_url": "",
          "post_date": "09/26/2020 13:38:49",
          "content": "<p>yes<br>\ngo ahead :D</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1027965,
      "author_name": "naim99",
      "author_url": "",
      "post_date": "09/26/2020 13:42:54",
      "content": "<p>Good job, continue like this ! BRAVO </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1008634": "What is **EfficientDet** and how does it work?\n\n\nIn short, it is a recent (first submitted at the end of 2019, accepted in CVPR in 2020) very efficient (surprise surprise) object detection model.\n\nIn more details, it is a family of models designed by researchers from **Google brain**. The interesting thing is that some parts of it were designed using automatic **architecture search**, i.e. there was a meta-model that was trained to find the best hyper-parameters of the trained model automatically. Let's dive in more details.\n\n\n\nFirst, let's start with the model's **architecture**: \n\n<img src=\"https://camo.githubusercontent.com/0ebd0147a2340c5ef6c3e30f2e51ab0cc812e515/68747470733a2f2f73756e392d33352e757365726170692e636f6d2f633230353632382f763230353632383732362f64323962342f67546a705534676a327a632e6a7067\" width=480>\n\n\nAs you can see, there are **3 main** building blocks:\n\n1. a **classification** backbone block\n2. a **multi-scale features fusion** block\n3. a **class** and **bounding box** networks block\n\nIn more details, we have: \n\n* an **EfficientNet** (**Net** and not **Det**) backbone. This backbone was found using NAS (neural-architecture search) through the use of the MnasNet framework. More on this in the next section\n* **BiFPN** as a multi-scale fusion layer: once we have a classification network we need to efficiently extract features for the detection step. One extra challenge is to do so for multiple scales (learn more about this [here](https://d2l.ai/chapter_computer-vision/multiscale-object-detection.html)). These two challenges are solved efficiently using the BiFPN layer. Again, more on this in the next section.\n* **Two heads**: finally, everything is connected to two output networks. One for predicting the most likely label and one for predicting the detected bounding box (if any).\n\n\nTo experiment with the model and learn more, check this [colab notebook](https://github.com/google/automl/blob/master/efficientdet/tutorial.ipynb), the original [paper](https://arxiv.org/pdf/1911.09070.pdf), and the original [implementation](https://github.com/google/automl/tree/master/efficientdet). \n\n\nIn the following sections, we will explore each part in more details. Let's go!\n\n## EfficientNet as a Backbone\n\nThis is the first part of the architecture. It uses the **EfficientNet** architecture and includes pre-trained weights.\n\nHere is a look at the overall architecture:\n\n\n\n<img src=\"https://miro.medium.com/max/1395/1*O1TZKZjafr9qqkxHxkJmAQ.png\" width=480>\n\nAs you can see, the interesting thing is that many elements of the model have been optimized using neural automatic search: **width**, **depth**, **number of channels**, and **resolution**.\n\nTo learn more about this **EfficientNet** backbone, read the original paper [here](https://arxiv.org/pdf/1905.11946.pdf), the blog [post](https://ai.googleblog.com/2019/05/efficientnet-improving-accuracy-and.html), and the implementation [here](https://github.com/google/automl/tree/master/efficientdet/backbone) (from the **efficientdet** repo).\n\n\n## BiFPN as a Multi-Scale Fusion Layer\n\nOnce we have a classification network, one important question to ask is: how to move from the **classification** model to an **object detection** one? This is cleverly solved by using features maps (i.e. intermediate learned representations). Another question is how to detect multiple objects having different sizes (for example a small cat and a car on the same image)? This is solved by taking features maps having different sizes. \n\nIn most modern object detection architectures, both problems are solved at once using an FPN layer. BiFPN, as its name indicates, builds upon the ideas of the FPN layer and adds one simple idea: instead of simply aggregating the different representations in a top-down fashion, it takes the PANet approach (i.e. top-down and bottom-up connections) and optimizes cross-scale connections.\n\nOne last trick is to use a weighted feature fusion instead of the unweighted FPN approach (the weights are learned as well) and normalize these weights using a fast normalization procedure (i.e. normalize by the sum instead of using softmax). Here is one example of a BiFPN computation from the paper:\n\n\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F612b65f04d5732669dbe04d5f0fc907d%2Fbifpn_computation_example.png?generation=1601114628541834&alt=media\" width=480>\n\n\nAll these tricks make the BiFPN layer both efficient and accurate. We will see one additional trick applied in the next and last section. Check the following graph to learn more about other FPN variations (including PANet and BiFPN):\n\n\n<img src=\"https://miro.medium.com/max/840/1*funGGmYqT6f2VkRnhqd2rA.png\" ![width=480](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F1ebba71862ecd6548995a9be7e11b9dd%2Fbifpn_computation_example.png?generation=1601114578505496&alt=media\" width=480>\n\n\n\nFinally, to learn more about FPN and BiFPN, check the paperswithcode [page](https://paperswithcode.com/method/bifpn), this [blog post](https://towardsdatascience.com/review-fpn-feature-pyramid-network-object-detection-262fc7482610), and this [one](https://medium.com/@jonathan_hui/understanding-feature-pyramid-networks-for-object-detection-fpn-45b227b9106c) as well. \n\nLet's move to the next step: the prediction networks.\n\n\n## Two Heads\n\nThe third ingredient to the mix is the **two heads network**:\n\n- one for predicting the coordinates of the **bounding box**\n- one for predicting the **class** (the probability of each class to be more precise)\n\nEach of the two networks take as input all the outputs of the previous **BiFPN layer**. Then we have two fully-connected layers before getting the final values. As simple as that!\n\nWe close this section with the last trick: compound scaling.\n\n\n## Compound Scaling\n\n\nOne last ingredient to the mix is how the network is scaled. Indeed, this is a trick often used to improve the performance \nof networks. The most common way to do it is to scale the backbone's width and depth. \n\nIn the EfficientDet model, the EffincientNet backbone is scaled reusing the same values in the original network.\nThe interesting additional thing is the scaling of the BiFPN and prediction networks.\nIn fact, instead of using only one BiFPN and one prediction networks, these components are repeated. Finally, width (W), depth (D), and input resolution (R) in the different sections are scaled using one single parameter $\\phi$ in this fashion:\n\n\n\n- 1. $$W_{bifpn} = 64 . (1.35^{\\phi})$$ and $$D_{bifpn} = 3 + \\phi$$ for the **BiFPN** part\n- 2. $$W_{pred} = W_{bifpn}$$ and and $$D_{bifpn} = 3 + \\phi$$ for the **prediction** part\n- 3. $$R_{input} = 512 + \\phi . 128$$ for the input image resolution. Notice that it should be divisible by 128 since we take features from levels 3 to 7: $$128 = 2 ^{7}$$\n\n\nHere is the table for the different **EfficientDet** variants: the bigger phi is, the bigger the model:\n    \n    \n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F4b349ddd45bddd490ae3bc5079cc0e25%2Fefficientdet_scalings.png?generation=1601122425468830&alt=media\" width=480>\n\n\nTo finish this section, notice that the **1.35** value in the first equation has been optimized using grid search (i.e. different values are tested and the one giving the best score is selected) over [1.2, 1.25, 1.3, 1.35, 1.4, 1.45] values.\n\n\nThat's it for the model's architecture. Time to move to the application!\n\n\nThat's it! I hope this (long) explanation was useful. Let me know in the comments if I have missed something important. \nThanks.",
    "1008749": "Loved it !!\nThanks for sharing !!",
    "1008762": "Glad it helps. For your information, this isn't yet 100% finished so stay tuned for more.",
    "1008787": "Yeah, would like to learn more from you.",
    "1021556": "thanks for sharing. I'm newbie. It's very useful",
    "1027756": "I am glad it helps! Check it again, I have improved some sections and added missing ones.",
    "1027841": "[EDIT 26-9-2020] Finished the remaining sections (BiFPN and compound scaling) and cleaned up few things.",
    "1027898": "Good work ! keep it up",
    "1027935": "Cheers! I hope it was of some use. ;)",
    "1027943": "yes\ngo ahead :D",
    "1027965": "Good job, continue like this ! BRAVO",
    "1028564": "Good Job! Still learning.",
    "1028799": "Keep learning!"
  },
  "source": "meta"
}