{"cells":[{"metadata":{},"cell_type":"markdown","source":"#  Stacking Ensemble Machine Learning Model","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"This is a beginner's introduction to Ensembling Machine learning models using the technique of *Stacking*.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"![image.png](attachment:image.png)","attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAApkAAAG8CAYAAACVJr5KAAAgAElEQVR4Ae2dDZajKhBGs64sKOvJanozsxjnoEGrClRUBNTb57wXVKifWyhfq515dfxAAAIQgAAEIAABCEAgM4FXZnuYgwAEIAABCEAAAhCAQIfIZBJAAAIQgAAEIAABCGQngMjMjhSDEIAABCAAAQhAAAKITOYABCAAAQhAAAIQgEB2AojM7EgxCAEIQAACEIAABCCAyGQOQAACEIAABCAAAQhkJ4DIzI4UgxCAAAQgAAEIQAACiEzmAAQgAAEIQAACEIBAdgKIzOxIMQgBCEAAAhCAAAQggMhkDkAAAhCAAAQgAAEIZCeAyMyOFIMQgAAEIAABCEAAAohM5gAEIAABCEAAAhCAQHYCiMzsSDEIAQhAAAIQgAAEIIDIZA5AAAIQgAAEIAABCGQngMjMjhSDEIAABCAAAQhAAAKITOYABCAAAQhAAAIQgEB2AojM7EgxCAEIQAACEIAABCCAyGQOQAACEIAABCAAAQhkJ4DIzI4UgxCAQDqBv+7zenWv/r939/2XPpKeEKhDgDlbhzter0gAkXnFqhEzBO5C4O/zE5iD0HyjMhuo7L/u+/bC//f5+WsgrhNCUPPv0yVlqca8OubsCXXB5G0IIDJvU8qHJ/Lv273HO2JygeTu2HkzQ97Rkcxle2XhNgv2XbXMeTXIbTkiMPvzaqWOYxhzcyJ1/GioTEPNv8QY1ZhXx5wtUyq8XJMAIvOadSPqkcDcoiaFDncbRlxBQ4uKTQvmrLDX7Jcfhcv68QtBUJ7SO4yAGmrn6rkuwP593+qu9DTWzwdr48Dcy8VF5Wvjm3PCnJ0jw34IWAKITEuE7QsRkBd7v5DFP3mkNVfWAwv9JpHp6oKInKtCK/u1UEwVXV3XKbEWPwfD+h+Ye7mAqbg35JvLP3YgcHMCiMybF/jO6f197GIWWSR+QgiROTcTDiz0RmSGd0EjvwS8vx1/2zNXi/r7lcjcUCt9Lobn4XDc7j8w93KhQmTmIokdCEQJIDKjWNjZPAEjcF4bFsQxN7XACMEaqqV+iFyAfRe9uK49lteLqn+c6G2Ncf0a0p/vO5en7Ovt2dhe/sDPfnDcvNNqutvwus7UYK6/9aP6KRuxO51xZo6HsmOikzxGdio/4UvG0BvVPrUffUza1v1GyOMjZP+LjuWhaxraj9o1+UY3t8zvub4/Zj72qJ/OxJwQcMBA1cbW1tgXfZddxcepMSpvI4LVsZn5Yu7Oy3nn/dhcl1jK8XJuTW0RR7wY7IVAUwQQmU2Vg2BSCdgLt7+gp42P3GETC5e/oFubcgH4fOf+0OjVWTHXx6QWLCFof361r/X4dH+n96b34T5/C+OFGLcMfd7+c2kx7HOS4mxR9Jl4ZPDKhllAV5j1cUpbA2jxlUghZ5+b+xyHyhg+3+Avq8d+e+IRY97f73xsri6ir4zTtVdroSa+4R2Z2yr/Ts8f63vdvxVzpo4qtmEjee4tMBnjHAskHK2MG4eofkJkyjnh+InzRv9ypXNV5+Gma0RazWzdRMY0IdAkAURmk2UhqGUCdlETi8PyQLecBiJiXKyCxVjblQvI/BgnbPTCoxeluPCZRERqfHtjmwTL2kI/LsRzTM1CvNRf+xKxKxuaWxrvpTG/Y8qH5y/GRY/7fpMY3RNP2ruKk6/5eSWYzdWj3586f5xPYVOJrTCeaX7GnUfZSGFmhun5EPrzcylqNzhPRS2dn5V6OsZjPipvz8MwtHko+9p3Wrwu36Vxv2PKj2ekxxmsbEKgOQKIzOZKQkDrBMxv/XYRWDKgFpXwrqNd/MbFKHq3R1zwjV05ztr0C2gf5m8hGfct2LGLp/QRLm5+wYwsuoqXXlDHOJYY+mNmEVwaq+Obi03wdD5+LAK7s35lLsZWJ+aMyj/CR97l9LnuimfKQYrHMR+Tx9BnYqOZTWJXhhS0zfyxd9XtXJRzyNlSPi2nwJncIfgGInDKSY6wv/CNXGSnzXPATRsvyIZPZXfxfBvi1OMjsau66Xmm+PUcxHFTm4n9znkrOdGGQKMEEJmNFoawlgiYBW3DYqgWkOg4ecHXj8n0AiIWjz5UPS6+gISi1map4lOr49BTxSDiV/vlHaqfA2VXHddxR1zaEKdttdgui6DZ+JQNy3RypVs65jhra0vMGcGtt6tiEHe5tNOFrbl4rMi0MVlBZAWNiHlO+JqoVJ1tnkOy+k6+6aPqZI4ZV5FNzUEK674dTC7dPzgc8TDt0mPjc2D9fPO/yIwCXwnBsF69fzVfdB/Fz9yttKI6HrO21y39cjTBoAWBZgkgMpstDYHNE9CLr3pfan6Qu0+jFtjpIq8H6YViWvjV/mABnrOtY11eSOdsiPjkIihiWI5tSexon8vxiThcUy2254lMlVtwl0yLQiWy/CJv4rR393QedpE3Odu7fSvxBCLGmFPxBvC31kb33zq/h5JO7/amn1cmKXOeSbGpY9LxBukLs+lzYMv5Zs+LhTugIpal+aLiFOfnMFznK1moeZA6b2VMtCHQKAFEZqOFIawlAvpird4tWxom7wos3BlSC4W466f2py4gRuAsLaTqrkVEvMjFum+LGJZjs4upFFKa5XJ8Bu6G3NQiKuJeWrD1MS0AJAu5WK8znH5pGLNReUg2Y4+hofolxiN/KRBzyVtWXAL4W2uTJrDUXDExqWOyTj7gjZ8qv35OS/4J+W1lbvoHSG38qj7v7q3+OU0ZqxiofOj5ssxP53t43oqQaEKgVQKIzFYrQ1yLBPTipS/08wP1Iqwv8tOouYVibv8wcmYBUQvS8t0+e6dVCqloW6ygy7HVFpmau7qTqPjIOmqeLv+pXvrYtN+8U2iFuuA1VdvekZUxyF7aZ2o8Ze9kas6Si8pEfBOBvVu5Oo+kocS2sunv0vVjNdOwPPp4EnM1n9bON3tefLo/M17NVZ+v6qPni8o1EOk6H1kfNS513vp4+IRAwwQQmQ0Xh9AWCKg7EPrdyflR+iJvF9hhnOkjVj61ECQvIGkLf8y3XITmc/qNlMLB3J1yPbQol3dodL4i3TWXyY/LtW+z8M8t2HP7+6h0zBMnwTqoz0I6i75+4xb7zMUTETEmDMUmgK/tBoeNreCXlCgDbdOKqOU5HjhM26HO1Q1zbxdzMQfULyYzoUZiUwxiTzwW4lJjA/6a/eF5O5MSuyHQEgFEZkvVIJYNBPRiMtzpkwvYz9RvQfAXdLUIRBYhteibBUaNTV5ArMAzIuv3CH8uvpiw8HHIY37fwME9yhUs1EJq/xhCL3xWdCwWRC22Ni83MlIjy03ZEHeF1H5t29bIs1t/vC6YyMSULxHDbJ/EeHoEn/HL2FVNfrZVLrKg/XFdm+CwjO/XtvNgZBPzZ+a366LG21pF/PW7PL9Yf3/M351TfXR+wdwzY2X+ips5j+0xOc7PyZGLOjf8/DBxyXNpgNS9fT7qzuwaP213jMHkOZ3D/pUMH9dcAdgPgXYJIDLbrQ2RrRFQC4S/IM98jitNRPiMC4YZqxbEnQuIyyEhznHBiQmzmfjGlKw4mOnvFy85bgjP5P0bP8U0U4jVxdHajQg4ZUMeT6/TFGfqGLNoz8Yg8061LR/r29obv/1hwcgWxvwBTXBYhje20+OM3ck/JDJX5p2bfzYHKwj9HB1qmp7LNAcsc8FXxDf2V+emqI/ab34xW5gvy/xmRGbyOS/iG+tNAwJtE0Bktl0folsjYBcDsZD4Bct9jouKs5c0Jryg71tAhgTU2EiMKj61iMUXycUvc36/xZ0WM96u8gs8VEyxOiTF6f2HPHuTyoYUmUbUR5j5+qo4k2prXq9YiEGmvVbD9XhCBkpkBbXRoiQ4LIOT7SQGYSzOhMrR/JIlXai24ufrHX6qOnkDM7H6viqe1Dlg84iM8/b1tUAzUbWRdyxVvgtzNuCn6znG4FjMcPBzavwMbHqQfEKgTQKIzDbrQlQbCcwvRnoRkGb1IjItinOLufIRudhLe2oB8U7V4rTuT9obFxm3YEYCjMVmx0eG+cgii9w8NzFo/p9J9At7hNM0vlcDQhBHfEaY9WzFoixZTzlrweB8KkbyEajyEYlBBqz6DjVcikeLhzCmKd5YXbUoWayfjPHXVrZ9PSJ3E+VQxWitdnLg0t24tcBFLYd5bmqwlbmPKzLO2VfhKN+2PuZOqh+o7OpY1/jJmhyetz5PPiHQMAFEZsPFITQIpBJYW9xS7Vy6nxAMcgEfc1oQB2MfGhAoTYB5W5o4/goSQGQWhI0rCJxFAJHpbsZOd4fDP7LRdwXD42dVBrsQWCbAvF3mw9FrE0BkXrt+RA+BngAi0z4Ol4IzbEfvdDKXIFCBgDp3xWsN6hWZ337mbYUC4fIQAUTmIXwMhkAbBNRCteldujbizxOFeYdubsH279blcYoVCBwkwLw9CJDhDRNAZDZcHEKDQDIB8V5X7Ktpku3coKMS3Epo2j/suEGypHAbAszb25SSRAQBRKaAQRMCEIAABCAAAQhAIA8BRGYejliBAAQgAAEIQAACEBAEEJkCBk0IQAACEIAABCAAgTwEEJl5OGIFAhCAAAQgAAEIQEAQQGQKGDQhAAEIQAACEIAABPIQQGTm4YgVCEAAAhCAAAQgAAFBAJEpYNCEAAQgAAEIQAACEMhDAJGZhyNWIAABCEAAAhCAAAQEAUSmgEETAhCAAAQgAAEIQCAPAURmHo5YgQAEIAABCEAAAhAQBBCZAgZNCEAAAhCAAAQgAIE8BBCZeThiBQIQgAAEIAABCEBAEEBkChg0IQABCEAAAhCAAATyEEBk5uGIFQhAAAIQgAAEIAABQQCRKWDQhAAEIAABCEAAAhDIQwCRmYcjViAAAQhAAAIQgAAEBAFEpoBBEwIQgAAEIAABCEAgDwFEZh6OWIEABCAAAQhAAAIQEAQQmQIGTQhAAAIQgAAEIACBPAQQmXk4YgUCEIAABCAAAQhAQBBAZAoYNCEAAQhAAAIQgAAE8hBAZObhiBUIQAACEIAABCAAAUEAkSlg0IQABCAAAQhAAAIQyEMAkZmHI1YgAAEIQAACEIAABAQBRKaAQRMCEIAABCAAAQhAIA8BRGYejliBAAQgAAEIQAACEBAEEJkCxlOar9er47/6DJ4y38gTAhCAAASeSQCR+cC6IzDrC0xXA34gAAEIQAACdybASnfn6s7khshEZM5MDXZDAAIQgAAEshFAZB5A+fd5da/P3wELdYZKkVkngud6hf1za0/mEIAABJ5GoDGR+a/7vrcKtz1j8pQZkZmH45OsIDKfVG1yhQAEIPBsAsdF5t+ne73e3fdfDpB7BOOeMTli7brNIjMrq/05IHT2szs6EvZHCTIeAhCAAASuQqAxkXkVbEOciMxr1auFaBGZLVSBGCAAAQhAoASBQyLz3/cdfBWOf0XRCbD3999wt899ZY4/0P11H/kVOu9vJ2+CWuEW2FG2BkR7xnQ2jl9MLua5n96Pj/3zF7mTOZ/bEqsgFsNkLp69+xE6e8kdHwf74wyxAAEIQAAC1yBwSGT2Kc48AvaCTIs292hbPloPH3XHBKNbmEc7/77dW2534WPrwHcwZhCDo82f4Jy2w+INNj+d/zMf72MSz+u5dVFWCePCcA7tQegcwndoMOwP4WMwBCAAAQhciMC5IjPhjlwv1kS/fnu86xkKSMd2rY89HozpRacUu6FNXUMrSoejMT9yXH9c5BYXmXKEsCvHhV0O7UHoHMJ3aDDsD+FjMAQgAAEIXIjAuSJTiMWJiXmk7B4/C0FlhZvddnbsvrXtcIwVjXZ7irZv/e6E2nSs3+Cxt8ltXmQuMzHRHN5E6BxGuNsA7HejYyAEIAABCFyMQGGRGYq5XqgVF5m/x/T+/Ur3aRWkLGSSyFzPLS4yE8bJWDK0EToZIO40Afud4BgGAQhAAAKXI3BcZEYePTsK4V2+fmfwdUdVRGYf8/R+5XrVBiFodajKMfK+pc2ti7FKGbce4KYeCJ1NuLJ2hn1WnBiDAAQgAIGGCWQSma9uUYB5APaOYC+wKjwu/8UhF3zXPvSHPwm5DSLTsEoZ5/ll+pR5ZzKJmUQCsE8ERTcIQAACELg8geMi09+1/D169mJT3eUTmPRX+Xy6r/sapOKPy92dSf2HP1EBKOLuOv2I3QnSPhefcNcN2+Mj+DA3Z67nYlitMVFhZNhA6GSAuNME7HeCYxgEIAABCFyOQBaRebWse1EnhO0Qf/hu5NXySo0XoZNKKn8/2OdnikUIQAACEGiTwHNFprmTOdxNNHc326zZ4agQOocR7jYA+93oGAgBCEAAAhcj8EiR6WokH1sPC/8zBKbLHaFT7yyFfT32eIYABCAAgbIEHisyy2JuyxtCp149YF+PPZ4hAAEIQKAsAURmWd5NeEPo1CsD7OuxxzMEIAABCJQlgMgsy7sJbwidemWAfT32eIYABCAAgbIEEJlleTfhDaFTrwywr8cezxCAAAQgUJYAIrMs7ya8IXTqlQH29djjGQIQgAAEyhJAZJbl3YQ3hE69MsC+Hns8QwACEIBAWQKIzLK8m/CG0KlXBtjXY49nCEAAAhAoSwCRWZZ3E94QOvXKAPt67PEMAQhAAAJlCSAyy/JuwhtCp14ZYF+PPZ4hAAEIQKAsAURmWd5NeEPo1CsD7OuxxzMEIAABCJQlgMgsy7sJbwidemWAfT32eIYABCAAgbIEEJlleTfhDaFTrwywr8f+qp7/Pq/u9fm7avjEDQEIPJgAIjOx+FIcuPaVf2QuufOouSD2vl+vzuXX6pp8JvvctcReGwRqnlNtECAKCEDgqgSurZYKUpfiINYuGMphVzL+w8aMgTIL4r/u+zZ3d/59u/fr3X3/+YAiffyhip9nsq+YVpuu/z7dS82JzGGebf8XbplzKjMbzEEAAhDoug6RmTgNpDhIaSeardJNxp87gDILYigg/33f3ev16aaHimGf3LnusXcm+z3x3HrM2SLwbPu/4pQ5p249E0gOAhCoRACRmQheioM97UQ3RbrJ+HM7rLUghiIzd2Z57J3JPk+EbVgZ6jm8+tAzM+8/xOaZ3BeMF69QuH7v779O9Xl/u/EmeNd10pYnIvepsQuvaPRjjO2u++s+Ih6/Pc4N01/6dbHY7cV9v9j0L2A+Iz4hAAEInEugeZE5XnjHi6VYeCrti5VkT5wxOyX2yVj3+usXupH/dAcxXACHBXX0aRbQrn/MPdVUaYmFY5Of3x3LMZZX9/r5mPpMWc7F7Xq4Y058jH1UMJONI62Rw8Xf6z3CYG3sIOCmOTWKMFGP2dqKPt3Mncawvr85KsYesa/yi8XQ7/P5ufm7/JqHjcVuO392X3TbnnsqUDYgAAEI5CeAyJTiJLGdUgYpJlLbKXZz9JHx7LHXL2Bywfr79OLM2dKL29oCGrmjM9pdOmb9dL+7Un7hHrLSsYRjbB799msQmnu4pIw5yj7Fx1If73+pT91jtu6/aIxYs7V1vYJ9ZozPK+g3DFavW8T6BPtm7Hs/w2eYT2BHDxjyGM+DMK/YeLUveD+5cyeIeWfZOGUTAhCAwAkEEJmJwtIvzu5zz48cn9re4ydljPSf0l/1+d1dFDd91GG12Kkjw0Z/3C+gS7aWjkUERXj3yyzOsUXW7FOxRWLPsesQ+4MBSN++fdBk/uGmJqMDMx9i8yzYNyMCg37Oiekb6xPsM2PGWE1DjwtF53inVl6L/DkSmeva3uBM7evjmp4O+Fqf+kdQJmc2IQABCDgC+xRTQXbTBbL5UDdRkXmltjc5WOgs/S10ix9aWVjVYtdbGBZV6dM/znaH+/794iofGQ6uV48JpbsqMhMW3jD2OIIjeyWHI3b2jpX+ZXuvvezjbigyBwH7u8vu5qEQkF5gutc0/E8/D0UfOy/tthun9q2co94PnxCAAATOJtC8cmtyITypKjLXlPbeMKTtzTZWFjC12P3+wGFpAfX++3FObIrFdemY9pPwuHwlbufL2vT+c34eYp8pEBmDbGcyf8yMuWM5GuvrN70OEatVsG9GsAb9+qfJ+tsJYn2CfTP2x5jHxvDesPudyNmQ54O9g+qG9H7EeWD92u1xjP+la47hGA8NCEAAAmUIIDLLcN7lRQqAWHuXUXf7WjyW22xjZQFTC2BE2NkFVPlfsm2OKT8RkeDsqj5mvPL721D9Yx0y7DvEPoN/aULGItuyT412Xwf1dVS/u+FeRI31nu5+D2Ni350afjF/2Dev/Riz/k7752v+yMe/Kyli7M8Z/cuWnZfDXful3H9/DCeE6nDHdBLpsRjZBwEIQCA3AURmbqIH7MmFPqW915W0vd1GZAFzC+NPAKgF0Qo7u4CKcX0c/fHf4rl0zArIUXToRVTF0kXi7u+0TmN0/+1kUkYcY5/iYXsfGZNsb7eUb0RfC/HLkLr793Oj+nz+hj/+EkLUdZN9/CFfZ3nMz1+ZgT0+CMXpm1jn7EsbY/t3LsTu1A+i0b9D+em+7jtfhUD08Y62TF4u9jC233wfGU6iVNqhDQEIQOBMAojMM+mu2JYLemp7xWTSYekraUDQyS5g80ItaQEdF0JxR8cupOp7Bc1dyiSR6ZKwceuFN7aYB6kf3HGc/cEAFobL2GR7YcglD5Wo8yXBEDQEIACBzAQuJTLlwlezvacGe+Ld4ydljIwlpT998hG4AnsZo2zno1DXEiKzLn+8QwACzyGAyBR30eSCutROmR5L4+eOpdjN0Uf6z2EPG+kErsJexmnb6dm22ROR2WZdiAoCELgfAURmJpFpF+KU7VrTScZWK4an+r0aexmvbT+1huQNAQhAAAJpBJoXmWlpnN9LLrDOm9xObZ8fZZoHGW/aCHrlInBV9jJu287FBjsQgAAEIHAvAojMxHrahXVtO9FslW4y9ioBPNjp1dnL+G37wWUldQhAAAIQiBBAZEagxHbZBdVux8a0uk/G3mqMd43rLuxlHrJ917qRFwQgAAEIbCeAyNzA7C6L6V3y2FC6Zrrejb3MR7abAU4gEIAABCBQjQAisxr6eo4RA7DPTUDOKdnO7Qd7EIAABCBwHQKIzOvUKlukiIBsKDcbujt7mZ9sbwbFAAhAAAIQuDwBROblS7g9ARb/7cxyjXgCe5mjbefiiB0IQAACEGifACKz/Rplj1Au/NmNY3CRwJPYy1xtexESByEAAQhA4BYEEJm3KOO2JOSCv20kvY8SeCJ7mbNtH+XJeAhAAAIQaJcAIrPd2pwWmVzoT3OC4SiBJ7OXudt2FBY7IQABCEDg0gQQmZcu377g5QK/zwKj9hKA/fK/lrWXK+MgAAEIQKA9AojM9mpyekQIndMRzzqA/YRGspDtqQctCEAAAhC4MgFE5pWrtzN2FvSd4DIMg30IUTKR7bAneyAAAQhA4EoEEJlXqlamWFnIM4HcYQb289AkG9meH8ERCEAAAhBomQAis+XqnBQbC/hJYBPMwn4dkmQk2+sj6QEBCEAAAi0RQGS2VI1CsbBwFwIdcQP7CJTILsnJtiPd2QUBCEAAAg0SQGQ2WJSzQ5KL9tm+sK8JwF7zWNuSvGx7bSzHIQABCECgLgFEZl3+VbzLxbpKAA92Cvt9xZfcbHufRUZBAAIQgMDZBBCZZxNu0L5cpBsM79Yhwf5YeSU/2z5mmdEQgAAEIJCbACIzN9EL2JOL8wXCvVWIsM9TTsnRtvN4wAoEIAABCBwlgMg8SvCC4+WifMHwLx0y7POWT/KU7bxesAYBCEAAAnsIIDL3ULv4GBbjegWE/TnsJVfZPscbViEAAQhAIIUAIjOF0s36sAjXKyjsz2Uv+cr2uV6xDgEIQAACMQKIzBiVm+9j8a1XYNiXYS85y3YZ73iBAAQgAAFHAJH5wHnAoluv6LAvx16ytu1yUeAJAhCAwHMJIDIfWHu54D4w/aopw748fsnctstHg0cIQAACzyGAyHxOrcdM5UI77qRRhADsi2COOpHsbTs6gJ0QgAAEIHCIACLzEL5rDpYL7DUzuG7UsK9fO1kD264fHRFAAAIQuA8BROZ9apmciVxYkwfRMQsB2GfBmMWIrIVsZzGOEQhAAAIQ4A9/njgHWFDrVR329djPeZY1ke25/uyHAAQgAIE0AtzJTON0q14spPXKCft67Nc8y9rI9to4jkMAAhCAQJwAIjPO5dZ7WUDrlRf29dinepY1ku3U8fSDAAQgAIGBACLzgTOBhbNe0WFfj/1Wz7JWsr3VDv0hAAEIPJUAIvOBlZcLJu1XV4vBA6fe5VJemhuXS4aAIQABCBQmgMgsDLwFd0sLJ8fKic4W5gIxpBFYOi/SLNALAhCAwPMIIDKfV/Nqd+6WFuonHnvg1Lt8ykvz9PLJkQAEIACBzAQQmZmBYg4CELg/AcTm/WtMhhCAwHECiMzjDLEAAQg8lMCc2HwoDtKGAAQgoAggMhUONiAAAQhsJ4DY3M6MERCAwP0JIDLvX2MyhAAEChFAbBYCjRsIQOASBBCZlygTQUIAAlcigNi8UrWIFQIQOIsAIvMsstiFAAQeTWBOaLr9/EAAAhB4AgGudk+oMjlCAALVCCA2q6HHMQQgUJkAIrNyAXAPAQg8gwBi8xl1JksIQGAigMicWNCCAAQgcDoBxObpiHEAAQg0QgCR2UghCAMCEHgWAcTms+pNthB4IgFE5hOrTs4QgEAzBObEZjMBEggEIACBnQQQmTvBMQwCEIBATgKIzZw0sQUBCLRAAJHZQhWIAQIQgMCPAGKTqQABCNyFACLzLpUkDwhA4FYEEJu3KifJQOCRBBCZjyw7SUMAAlcgMCc03X5+IAABCLROgCtV6xUiPghA4PEEEJuPnwIAgMAlCSAyL1k2goYABJ5IALH5xKqTMwSuSwCRed3aETkEIPBQAojNhxaetCFwMQKIzIsVjHAhAAEIeAJzYtMf5xMCEIBATQKIzJr08Q0BCEAgAwHEZgaImIAABLITQGRmR4pBCEAAAnUIIDbrcMcrBCAQJ4DIjNtdHjwAACAASURBVHNhLwQgAIHLEkBsXrZ0BA6BWxFAZN6qnCQDAQhAYCKA2JxY0IIABMoTQGSWZ45HCEAAAsUIzAlNt7/mj4yrZhz4hgAEziNQ9ypzXl5YhgAEIAABQUCKOtsW3Yo1ZQzFnOIIAhAoSgCRWRQ3ziAAAQjUJSDFnW2XjEz6LukXXxCAQDkCiMxyrPEEAQhAoBkCUuTZdokgpc8S/vABgbsR+Pu8utfnr+m0EJlNl4fgIAABCJxLQIo92T7Xa9eV9HV2LtiHQA0CiMwa1PEJAQhAAAKbCUjRJ9ubDSUOKOEjMRS6PY3A36d7vd7d999JiZ9t/xc2IvOk+mEWAhCAAATOISDFn2zn9nam7dyxYu9mBM4WgWfb/5UDkXmzeUk6EIAABJ5CQIpA2c6V/xk2c8WGnXYJ/Pu+1asW9p3EmPCS+4Lxr1fnX2t0/d7ff53q8/528oantOUpyX1q7OvVx+rt+/7usx9jbHfdX/cR8fjt8Vwx/aXf0aZxZvuM/X6xvV6f7sy3OnknU1adNgQgAAEIjATGxW1ckIZF0+0/+iNtH7XF+GcQGAScFEWDKJNCc1ZUSfE1c6exH+vm+tg3r31VpVgM/T6f37/u+5aP9N22jO0nVMdYw23nz/KIbhvxquI8uHH8SnEwAIZDAAIQgEDbBKQgtO29kUs7e20w7kkE7F2+X+5GrFkR5XoF+8wYTzHoNwzu5N2+WJ9g34x972f4DPMJ7OgBwd1P299uD+ELYfrv273tu6ixfcbvkU1E5hF6jIUABCDwIAJSGNr2Vgxy/Nax9H8ggTkx1O/Xj7ynO5EDp0B8zYjAoJ8bbvrG+gT7zJi5aulxoegMHpe7u6zirqMeHxHTVmD3cU1PI6ZzUN4xnYt2335E5j5ujIIABCDwWALT4hQuWKlQpI3UMfR7MIEbisxBwP4ejzsBKASkF5juHVH/04tK0WefyDxPUPo45SciU9KgDQEIQAACyQSkULTtNSOy/1pfjkOgM3csRyL93Tn/HmPC3Tw3cEawWtE2dHV/aJTH/hjz2Bjes3SvVTrfUlDaO6huyGGROcdwjCd/A5GZnykWIQABCDyKgBSMsr0EIbXfkg2OPYtAL7KE4PN3++Tj8eGPg6a7dcMY8V6iQzYjtsK+4R/+HLEfq1Zv7/M1f+QTidE/6l64k7ke2++Ph4SNgeEkomMxHtmHyDxC76Jj5cWddvi4rxSTi04fwobALIG5cyc2QPaNHWcfBGIERiH4+8YDdffvN0D1+fwNX0kk/grbdZN9/KF+3+dPHZMC1scjx7rjg1DUXwQk+3j7frz6/Ale/ah86DGIRr9Gfbqv+/omIRB9vNKe9BuP7Sc0x2+MmAS5tJOrjcjMRfJCduTFnbY/gct/XmjKECoENhGYu65II7KP3E8bArUIxERbrVju4heReZdKbshDXtxplxeXnvmGktEVApck4Oe6/XTJyH2XTI6gb0cAkZm/pIjMA0yvOiG5uB8o+sGhsD8IkOGXIyDn/Fz7ckkR8C0JXHVNb7kYjYnM8Bvt1+HtGbNuNaXHVSekvNCn5EmffARgn48llq5FQM59275WJkQLAQikEjguMvu/eMr14ugewbhnTCqe5X6bRWZWVsuxLR2VF/ilfhzLTwD2+Zli8VoE5Dng29fKgGghAIFUAo2JzNSw2+iHyGyjDleKwi+q7pMfCDyZQIlzQfqgzfvnTz7fauV+aKXTf14/TGD/p/pOgLmvFuiFWOwfnPd/Pi/+HN9BsMItsKNsDdj2jBm/X8vH8fuMfR2CL86Yyy8G6zewKXJbYrU0zvvO+SkvtjntYmudAOzXGdHjGQRKnAvSB21E5jPOrLayPCQy+1RmHgF7QaZFm3u0LR+th4+6rXAL7Py+U0ra3T5m+ILVyYbdDos0xDF9YamPa/oOrfXcYt/g33UJ48JwDu2RF9tDhhi8mQDsNyNjQCECcm6Wbp+VYuk88BcXsmfVF7vtEzhXZIo7eXMoerEm+kUFo789+jOy1sced8PUvl6oSrFrjgfBxkWoshmM+dkUucVFZjiwtyvHhV0O7ZEXwkOGGLyZAOw3I2NAIQJybpZun5WizOMsH9iNE4B9nMvT9p4rMo04HOD+/pkm+ZhaCCor3Oy2s2H3rW2HY6xotNtmGiz9E1Qqx+Xc5kXmyjgTztFNTv6jBPePh/1+dow8l4Ccm6XbZ2Um8zjLB3bjBGAf5/K0vYVFZijmeoFYXGT+HtNLoavEopkGSSJzPbe4yEwYZ8I5usnJf5Tg/vGw38+u1ZH2l9xW43xiXJxv9aoO+3rsW/J8XGRGHj27BKMX3sj7m1VEZh/z9H7lekEGIWh1qMoxIbcuxipl3HqAm3pw8m/ClbUz7LPibMKYug40ERFBeAKcb55E+U/Yl2feosdMIvPVLQown7m9I9gLrNfiP/geu4DbfWvbzr3q84tDngSuPf0hkA94+uzHvyZhOmy/uvEPfxJyG0SmYZUybgojS0vmncUgRpIJwF6givyCJY4eb55t/xehurYcjxoLGQlwvmWEudEU7DcCu2n34yLTC7jfo2cvNucuvPqrfD7d9/suLzI7d2dS/+FPVACqoutH7E6Q9rn4hLtu2B4fwYe5OXOjOH1NYnONiQojwwYnfwaIO03AXoA7WwSebf+Xyty1TmRKsxIBzrdK4Pm36euBb8xzFpHZWE6r4fSiTrwHOgwI341cNXTRDlx46xXuLuz1L0bijv4PbUx4yX3BePFLl+s3/hLnf2kz56u05asp9y3Z9/3dZz/G2PbfWzv9/jhcG8bamf7S72hzGty7s33Gfj4/8ZSkH8D/shAYa8Y/fpCF5xYjsN9C6759nysyzZ3MYVEydzdvWndO/nqFvQP74VyZXh3xomx8dcSLtzWhNXOnsRdkTnyN438ib9w2r7/8yhkIuRn7qvqxPv0+n9/699hav3bb+bP7ottGvKo42dhF4A7n267EGxgE+waK0EAIjxSZjnt/kR/vIrgvkH2GwHS5c/LXO/Ouz34QfELvDTCNWLMiajzn5EAzxlclNnb4ZgYv/ELRtsW+9zN8hvlE/YtB/XEhCG1/ux3E1r+Hba43sX3CJ819BK5/vu3Lu4VRsG+hCvVjeKzIrI++XgSc/LDfTWBODJk/YFsVWi6AzSJzEmZH7Nvcta1QdI53auUvpUdEZp937F9GmfKzMbK9jwDXun3ccoyCfQ6K17eByLx+DTdnwMm/GVm2AZdnf0ORqe6SOgEoBKQXmPKbJ3pRKvpokZpwl3VGXGebZBgaCVz+fBszuV4D9ter2RkRIzLPoNq4TU7+egW6PHtzx3Ik2QunjY+zZwSrFW3Oh30PNNYn2Ddjf4x5bAzfHOGe5DsbUlDG7rb2fo6IzDmGYzw0chG4/PmWC0QFO7CvAL1Bl4jMBotydkic/GcTnrd/B/a9yFJ/DR3+YY79Q7phjPxjnl45dm/xV+WeWtg3r33vR3728X6+3fdtHllbQegfdS+IzPXcf1+HJmwMd0wnkS5jo72fwB3Ot/3Z1x0J+7r8W/GOyGylEgXj4OQvCNu4ugv7UQj+3lNUd/9+Oas+n7/ge2VdN9nH/01Qv+/zp45Nf2k+AZVj3fFBKP5NHWbsqw5+4ycm9aPy4eAgGv07lOH33/p4vSn3uR6b/t7dJ/3hoeR0dvsu59vZnM6wD/szqF7PJiLzejU7HDEn/2GEuw3Afh1dTLStj6IHBEICnG8hk1J7YF+KdNt+EJlt1+eU6Dj5T8GaZBT265gQmeuM6JFGgPMtjdMZvWB/BtXr2URkXq9mhyPm5D+McLcB2K+jQ2SuM6JHGgHOtzROZ/SC/RlUr2cTkXm9mh2OmJP/MMLdBmC/Gx0DIbCZAOfbZmTZBsA+G8pLG0JkXrp8+4Ln5N/HLcco2OegiI0rEvBzv2Ts3qf75KcsAdiX5d2qN868VitzYlyc/CfCXTEN+xVAHL4lgVrzvpbfWxZxY1Kw3wjspt0RmTct7FJanPxLdM49Bvtz+WK9TQJy3rt2qR/pt5RP/AwEYM9McATKne0X5y1PmJIXyTOwyVxy26/5Rxu979/3NvrvXMyd31F7Z7I/GhvjIXAmATn3S11Dpc8zc8N2SAD2IZMn7kFkJlZdnjCxdqKZJrrJ+HMHVEZk/r7IWirJ4J8QjPTJnewOe2ey3xEOQyBQlICc/6599o/0d7Yv7GsCsNc8nrp1/ll+E7LyhElpt5y2jD93nLVE5vCvssh/lg+Rmbu22INADgJnXn9sfCV9Wd9P34b902fAkD8iM3EeyBNmTzvRTZFuMv7cDsuIzDDqUGSGfVrYcyb7FvIjBgisEZDngGuf+SN9nekH2yEB2IdMnrjn3DM8A1E5UVtpx9LaE1vMTol9Mta9/uT7j6/XdAcxFJl/3ef3nmTv9/3t/kmn/t+Mjr1LuXBs8mP/DehX5//96anP5HAubtfDHXP/BvfYRz6On0wcauVgfygABkOgAQLyPHDts36kn7N8YDdOAPZxLk/be97ZnYmknKittFNS2xNrit0cfWRse+z1IkyKxb9PL86crf7YKM6cAHx331FV2kfYgwAdu3d/3We0u3TM+um62J1MHUs4xubRb78GobmHS8qYo+xTfNAHAlcgIM8F1z7jR/o4wz425wnAfp7Nk46cc2ZnJCgnaivtPentiX2Pn5QxMpaU/qrP7+7iJAzVUSMy9TG3pYTdkq2lY96OCGJVZPb2pODtnDLt3q9pn4otDD3LnkPss0SAEQi0QUCeC659xo/0kdt+f70Q16Dc9pfs+V+IXX6VQlgKrz92JvtV53RohsA5Z3bG9O46UWVeqe1cWKW/zTb/Pt1LCDM7Przwmsfl7rH4eLfyJzr7R+WT2PM2pwvpzDFxdV0VmX3cr07mPrQn22HsPpJ8n9J/PqtYgsA1CcjzwbVz/0j7uW2XuF50nX36E/5yHO2TO9kd9s5kvyOcXUPukMOuxDMOyn9WZwzOmXpSkWWuKe29qKXtzTY2icxBYLr3HP1Pf2EWIlPtNwJ06Zi9wKeJzElQetvy09qUx3K1D7HPFQR2INAQAXlOuHbOH2k7p11nq8T1IiYgw2tdRIjmTnaHvTPZ7whn8xAZ/1x7s9EHDsh7Rp8AUBb3BPNNm5S5x9p7g5e2NtvY8hg7IkjnRGYfx5Jtc8xe4MMLr1kEzPhY3tZmrM/RfYfYH3XOeAg0SuCs8+Isuw5jietFrFyxa12sX+19Z7IvkZuMf0u7RGxX8oHIbKhaWyay67v3R/rZbuP3W7O8G+nE5O/RtbrwWmHnH1n7sWJcH4cUpUvHIhf42IVXxeIfO3nfg8Pus/iX8dvprI04xn7NOschcE0C8rw4cm2z2Uu79ljqdn8d+X37Bd+kkUrt+k8h7dyR21vb6dTu13O/UinEQhazkMtibmRuqe0cwUlf++z9hGbChXcQf/5dyE/3/b5n3skc+ojXLIc7BaMP/YK7FpBpf10+PnoaberH59bmPjbLo46zj9uXdmn7+cbnledCfKZv2yvz3zZy6N1fE+QvpnyTRjLGLexl3xbbc0kfiXXO5t32X0pkHilozrF7JsEe/3v8pIyRsaT0p08+Amexl3ZpIy7vMgeOnnmSw2Zb9kmMMbD2S6kSqEu2lo7teWrT29O/QLf+TRqyTi22TekXN4/Ev2j4ogcRmeNdrfSFKaXWeyZait0cfWRsOexhI53AWeylXdrp5zKs2maVfmbFe8r6xnss7JWv70S6hSKTb9KQmLawl31bbMu89rSP5LTHX0tjEJmZROaeSVRrIshYa8XwVL+wf2rlyXuNgDw3fHttzNpxb8d9bv7ZJDL5Jg3Ldwv7LX2tn6tuy5y3tq+U844z70rp5YtVTgJnVW6ntvNFc8ySjPeYJUZvJQD7rcTo/xQCZ5wbh2xueYwdEaTqcbkt4pJtc8zeMV39I0cz3rp229ZmrM/RfVvYb+l7NK6Wx0sOKe2Wc/GxITI9iZXPlILLPivmqh6+SpxVIZ3kHPYngcXspQnI88K1c/1Iu9tt8k0a25lNI7aw39J38nDPlmSx1r4CgXxn8xWyPRDjHYrt05e5+H18liEA+zKc8XItAmedF8ft8k0ae2fSFvZb+u6Np8VxMu+t7RbzicWEyIxRmdknJ8FMl0vsvksel4BtgoS9AcLm4wmceU6cafvxhVsBsIW97NtieyXVpMNH8kpy0GgnRGajhTkzLDnZz/SD7ZAA7EMm7HkuAXk+uHbuH2k/t23sLRPYwl72bbG9nGl49EgOobVr78l/Vl+bxyOilyfAIxJuKEnYN1QMQqlO4Ozz4Wz71QE2HMAW9rJvi+0lzEfiXbJ7l2OIzLtUckMe8qTYMIyuGQjAPgNETNyCgDwXXPuMH+njDPvYnCdwdfax+OW+re15Uvc+cs6ZfW9ml89OnhyXT+ZiCcD+YgUj3NMIlDgXSvg4DdDFDV+dvYx/a/vipcsaPiIzK85rGJMnzDUivk+UsL9PLclkPwF5Hrj2WT/Sz1k+sBsncHX2Mv6ldjx79noC553d3gOfzRGQJ0xzwd08INjfvMCkl0Sg1HlQyk9S0g/rdAf2MgfX5mc7AahtZ3b5EfLEuXwyF0sA9hcrGOFmJyDPgbMXbukreyIYXCQA+0U8jzmIyHxMqadEOfknFqVbsC9NHH+tESh5DpT01Rrn2vHAvnYF2vCPyGyjDkWj4OQvils5g73CwcbDCJSe/6X9Payci+nCfhHPYw4iMh9T6ilRTv6JRekW7EsTx18rBOTcd+0SP9JnCX/4mAjAfmLx5FaZM/3JhBvMnZO/XlFgX489nusSqDH3a/isS7kd77BvpxY1I0Fk1qRfyTcnfyXwXdfBvh57PNcjIOe9a5f6kX5L+cTPQAD2zARHoNzZDu9mCHDy1ysF7Ouxx3M9ArXmfS2/9Ui34xn27dSiZiSIzJr0K/nm5K8EnjuZ9cDjuToBf90pGYj36T75KUsA9mV5t+qNM6/VypwYFyf/iXBXTMN+BRCHIZCRAOdbRpgbTcF+I7Cbdkdk3rSwS2lx8i/ROfcY7M/li3UISAKcb5JG2Tbsy/Ju1Rsis9XKnBgXJ/+JcFdMw34FEIchkJEA51tGmBtNwX4jsJt2R2TetLBLaXHyL9E59xjsz+WLdQhIApxvkkbZNuzL8m7VGyKz1cqcGBcn/4lwV0zDfgUQhyGQkQDnW0aYG03BfiOwm3ZHZN60sEtpcfIv0Tn3GOzP5Yt1CEgCnG+SRtk27MvybtUbIrPVypwYFyf/iXBXTMN+BRCHIZCRAOdbRpgbTcF+I7Cbdkdk3rSwS2lx8i/ROfcY7M/li3UISAKcb5JG2Tbsy/Ju1Rsis9XKnBgXJ/+JcFdMw34FEIchkJEA51tGmBtNwX4jsJt2R2TetLBLaXHyL9E59xjsz+WL9WsS8OdF7ui9XffJT1kCsC/Lu1VvnHmtVubEuDj5T4S7Yhr2K4A43DwBOYdT2msJSRtrfbceP9P21lie1h/2T6t4PF9EZpzLrfdy8tcrL+zrscdzHgJyDudu54lwsiLjm/bSKkEA9iUot+8Dkdl+jbJHyMmfHWmyQdgno6JjowTkHM7dzp2yjC+3bewtE4D9Mp+nHEVkPqXSIk9OfgGjcBP2hYHjrioBOd9T2rmDlT5z28beMgHYL/N5ylFE5lMqLfLk5BcwCjdhXxg47pogIOe9bLvg7HbOgM+0nTPOO9qC/R2ruj0nROZ2ZpcfIU9+2i+1yJXkcfmJRAIQWCEwdz7ZYbKfPXZk+yy7R2J6yljYP6XSy3kiMpf53PKoPPlpIzJvOclJqjqBuWtLLDDZN3Z8776z7O6N50njYP+kas/nisicZ3PbI/Lkp43IvO1EJ7EqBOauKUvByDFL/bYeO8vu1jie2B/2T6x6mDMiM2TCHghAAAIQ2EhAigrZTjGztX+KTdfnLLup/p/cD/ZPrv6UOyJzYkELAhCAAAQ2EpBiQra3mNk7bs3HWXbX/HIcgc8cGAggMpkJEIAABCCwmYAUcLK92dCJdxyPxrUnF8YMBGDPTHAEEJnMAwhAAAIQ2ERACgjZ3mREdM5hQ5gbm9Iubd4/HycGjWIEEJnFUOMIAhCAwLUJzAm1o1lJu0dtyfHSLm1EppwbtMsQQGSW4YwXCEAAApclMCfQciUk7eey6exIu7QRmTnnFrbSCCAy0zjRCwIQgMDjCMwJs9wgpJ/ctrEHAQjUI4DIrMcezxCAAASaJCBFn2yfGWwpP2fmgG0IQEATQGRqHmxBAAIQeDQBKfZk+2woJX2dnQv2IQCBgQAik5kAAQhAAAKz7y+WQoPILEUaPxAoRwCRWY41niAAAQg0R0CKO9kuHWhN36VzxR8EnkIAkfmUSpMnBCAAAUFAijrZFl2KNluIoWjCOIPAAwggMh9QZFKEAAQg4AlIMWfbvk+NTxlLDf/4hAAE8hNAZOZnikUIQAACTRKQQk62WwnWx9RKPMQBAQgcI4DIPMaP0RCAAASaJ+DFm/1sPnAChAAELk0AkXnp8hE8BCAAgXkCVlT67fkRHIEABCCQjwAiMx9LLEEAAhBogoAXk/azieAIAgIQeAwBROZjSk2iEIBACoG/z6t7ff5SujbZxwpLv91ksAQFAQjcmgAi89blJTkIQGArgauKTC8m7efW/OkPAQhAIBcBRGYuktiBwNMJ/H261+vdff+dBOJs+7+wryYyraj02ydVAbMQgAAEkgkgMpNR0RECEFgkcLYIPNv+L7mriEwvJu3nYo04CAEIQKAgAURmQdi4gkCrBP593/rfrjbvJMaEl9wXjH+9Om/C9Xt//3Wqz/vbyRue0pZnJPepsa9XH6u37/u7z36Msd11f91HxOO3R3Fm+ku/o03jzPYZ+/1ie70+3VlvdY5xj74GHpIDbQhAAAItEEBktlAFYoBARQKDgJOiaBBl8o9fZkWVFF8zdxr7sU4QjX3z2lfoYjH0+3x+/7rvWz7Sd9sytp9QHWMNt50/yyO6bcSrinPnBgJzJziGQQACVQggMqtgxykEWiFg7/L94jJizYoo1yvYZ8b4DIN+w+BO3u2L9Qn2zdj3fobPMJ/Ajh4Q3P20/e32EL4Qpv++3du+ixrbZ/xu2URcbqFFXwhAoBUCiMxWKkEcEKhBYE4M9fv1I+/pTuQQaCC+ZkRg0M8NN31jfYJ9ZswcLj0uFJ3B43J3l1XcddTjI2LaCuw+ruGRtRaD8o7pXLTL+7W9ycfyKI5CAAIQaIMAIrONOhAFBOoQuKHIHATs7/G4E4BCQHqB6d4R9T+9qBR99onM44LSx+M+EZeSBm0IQOCqBBCZV60ccUMgBwFzx3I02d+d8+8xJtzNcwNnBKsVbUNX94dGeeyPMY+N4T1L91ql8y0Fpb2D6oYcFplzDMd40huIy3RW9IQABNongMhsv0ZECIFTCfQiSwg+f7dPPh4f/jhouls3jBHvJboIZ8RW2Df8w58j9mNwenufr/kjn0iM/lH3wp3M9dh+fzwkbAwMJxEdi9HuQ2BaImxDAAJXJ4DIvHoFiR8CGQiMQvD3tTjq7t/Pvurz+Ru+kkj8FbbrJvv4Q/2+z586JgWsD1+OdccHoai/CEj28fb9ePX5E7z6UfnQYxCN/v3GT/d1X98kBKKPV9qTfuOx/YTm+LVCkyCXdmJtxGWMCvsgAIE7EEBk3qGK5ACBhgnERFvD4RYLDXFZDDWOIACBSgQQmZXA4xYCTyGAyNSVRlxqHmxBAAL3JYDIvG9tyQwCTRBAZA5lQFw2MR0JAgIQKEgAkVkQNq4gAIFnEkBgPrPuZA2BpxNAZD59BpA/BCBwGgHE5WloMQwBCFyAACLzAkUiRAhA4FoEEJfXqhfRQgAC5xBAZJ7DFasQgMADCSAuH1h0UoYABGYJIDJn0XAAAhCAQBoBxGUaJ3pBAALPIoDIfFa9yRYCEMhMAIGZGSjmIACB2xBAZN6mlCQCAQiUJIC4LEkbXxCAwBUJIDKvWLWDMc8tjuz3/9Rgmc+DZWR4JQJz50mlcHALAQhAoFkCiMxmS3NeYHOLJPvLiEvP+bwKY/kMAr5u9vMMX9iEAAQgcAcCiMw7VHFjDnaRZLusuPS8N5aN7pUI+HrFPiuFhFsIQAAClyCAyDxQpqv+c3lysTyQPkN3EID9DmgVh8h6yXbFkHANAQhA4DIEGhOZ/7rv+9W9Pn8bAO4Zs8H8QldE5gIcDkUJIFSiWJrbKesk280FSkAQgAAEGiZwXGT+fbrX6919/+XIco9g3DMmR6xdt1lkZmW1PwcWzf3sjo6E/VGC546X9ZHtc71iHQIQgMA9CTQmMq8FGZF5rXq1EC3CpYUqhDHIush22JM9EIAABCCQSuCQyPz3fXfyguza/km3E2Dv77/hbt9LPgL/6z5u2//3/nbyJqgVboEdZWtIc8+YrjNx/OJxMc/99H583J+/yJ1MY1PktsQqiEWMm4vlyP6R/etQ+Y+E8NixsG+r9LIett1WpEQDAQhA4HoEjquMmUfAXpBp0eYebctH6+Gj7phgdBf/0c6/b/eW21342DrwHYwZxOBo8yc4p+2wkIPNT+ffFvU+pvdH13ProqwSxoXhHNojF9NDhhi8mQDsNyM7bYCshWyf5hDDEIAABB5G4FyRmXBHrhdrol+/7W+HRgSkq89aH3s8GNOLTil2Q5t6HlhROhyN+ZHj+uMit7jIlCOEXTku7HJoDwvqIXyHBsP+EL4sg2UNZDuLcYxAAAIQgMBI4FyRKcTi6DH2mFoIKivc7LazY/etbYdjrGi021O0fet3J9SmY/0Gj73do3WR27zIHPzLBU+NM+Ec3ZR+jtpi/DYCsN/GK2dvyV62c/rAFgQgAAEITAQKi8xQzPVCTQgxK9zstgvd7lvbDsf8HtP79yvdp1WQE6OuSxKZ67nFRWbCOBlLhjYLbAaIO03Afie4A8Mkc9k+XEA49wAAEQhJREFUYJKhEIAABCCQQOC4yIw8enZ+rfDrY4m8k9j3Ky0y+5in9yvXOQ1C0OpQlWNCboNY1Y/pY8LTMlmPb1sPFtptvHL2hn1Omuu2JG/ZXh9JDwhAAAIQOEogk8ic/qrcB6QEmN9p7wj2wkw/Urbj7LYzZfetbQdjfnHIRce1D/3hT0Ju0TuiKeM8v0yfMu9MJjGTSAD2iaAOdpOcZfugWYZDAAIQgMAGAsdFphd9v0fP/m6fFX4+Jv1VPp/u674GqfSdzP69UHNH0Yo9H/D4qR+xO0Ha5+IT7txTdfmVTmFuzlTPxbBKGTeGkaHBopsB4k4TsN8JLnGY5CvbicPpBgEIQAACGQlkEZkZ4yliqhd1QtgOTsN3I4sEU8EJi28F6D+XsD+HveQq2+d4wyoEIAABCKQQeK7INP8U5nA30dzdTCF4wT4swvWKBvu87CVP2c7rBWsQgAAEILCHwCNFpgMlH1sPi9MzBKbLncV4z6mSZwzs83C08xiu+bhiCQIQgEAuAo8VmbkAXtEOC3K9qsH+OHvJULaPW8YCBCAAAQjkJIDIzEnzIrZYmOsVCvb72Ut2sr3fIiMhAAEIQOBMAojMM+k2apsFul5hYL+dvWQm29stMQICEIAABEoSQGSWpN2ILxbqeoWAfTp7yUq20y3QEwIQgAAEahJAZNakX8k3C3Yl8PzRVTJ4OUdlO9kAHSEAAQhAoDoBRGb1EpQPgEW7PHPvEfaeRPxT8pHteG/2QgACEIBAywQQmS1X56TYWLxPAptgFvZxSJKLbMd7sxcCEIAABK5AAJF5hSpljpFFPDPQDeZgr2FJHrKte7EFAQhAAAJXJIDIvGLVDsbMYn4Q4IHhsB/gSQ6yfQAtQyEAAQhAoDECiMzGClIiHBb1EpTjPmCv/8UpeMTnCXshAAEI3IEAIvMOVdyYAwv7RmAZuz+ZvcxdtjPixRQEIAABCDREAJHZUDFKhcICX4p06OeJ7GXOsh3SYQ8EIAABCNyJACIzsZpycXTtK//IXHLn8fd5da/PX26zSfZ6369X5/KrFMJqnGeyX3VeuIPMVbYLh4E7CEAAAhCoRODaaqkgNLlIxtoFQznsSsZ/2JgxUEZk/uu+byNm/3279+vdff/5gCJ9/KGKn2eyr5iWci1zlG3ViQ0IQAACELg9AURmYonlYpnSTjRbpZuMP3cAtUTmv++7e70+3XQPFZGZu7Yp9uTcku2UsfSBAAQgAIF7EUBkJtZTLph72oluinST8ed2WEZkhlGHIjPs08KeM9nXzE/mJds1Y8I3BCAAAQjUJdC8yJQLVivtWMn2xBazU2KfjHWvP/n+o7yDGIrMv+7ze0+y9/v+duMTbee8f8w9vEfpjqt3KReOTX5+dywjPqY+U5Zzcbse7tj7+6//7GNVwUw2jrRysD/iP/dYmY9s5/aDPQhAAAIQuB4BRKYUJ4ntlDLLBTe1nWI3Rx8Zzx57vVCTYvHv04szZ0sLOycAl96THATopOX+us9od+mY9eO0qn1cHvbRsf2Oj/5+269BaO7hkjLmKPsUHyX6yDxku4RvfEAAAhCAwDUIIDITheXRhVSOT22fNYWk/80+fncXJ2GoLVghp48aYbdka+lYIGYTRGZvTwpefxd12heIZxt8hu1D7DP4P2pCxm/bR20zHgIQgAAE7kXgUiLzTujtAp2ynSt/6Wuzzb9P91J/xa0thCLTPC53oj5y9zBms7fV/xIwCUHvzfpZvZPZxz09lp8YTLatTe8r5+fkt/lTL0hbxi7bQUd2QAACEIAABLqua36le9JiJnNNae+dwdL2ZhubROYgMN17jv6nF3JCZKr9RoAuHbOCME1kToLS25af1qY8lqt9iH2uIDbakTHL9kYzdIcABCAAgYcRQGQ2XHC5oMfae0OXtjbb2PIYOyJI50RmH8eSbXPMCsJVkWnGx/K2NmN9ju47xP6o843jZayyvdEM3SEAAQhA4KEEEJkNFV4u5CntvaFL29tt/P6aW96NdGLy95KmEmpW2PlH1n6sGNfHIUXp0rE972R2kbg7d6d1+m5NFft2MEkjjrFPcnG4k4xRtg8bxgAEIAABCDyKACKzYrnlAp7azhGu9LXPnv3aoHmhNtxh9O9Cfrqv+ytwLzK9WBR/fCX/oKgXfUvHROfVO5l9ojZu/fgckdn1/ySnnB++vW+eMAoCEIAABJ5M4FIi0y94tT/3TJg9Me/xkzJGxpLSnz75CLTKXsYl2/kyxxIEIAABCDyNACJT3CmTi+tSO2WSLI2fO5ZiN0cf6T+HPWykE2iNvYxHttMzoicEIAABCEAgTgCRmUlkygU6tR0vyfl7ZXzne8ODJNAKexmHbMtYaUMAAhCAAASOEGheZB5JLudYuxDL7dR2zniO2JLxHrHD2O0EarOX/mV7eyaMgAAEIAABCCwTQGQu8xmPygU5pT0ObLAh428wvFuHVJO99C3btwZOchCAAAQgUI0AIjMRvVyUY+1EM010k/E3EdCDgqjBXvqU7QdhJ1UIQAACEKhAAJG5AfpdFui75LGhdM10Lcle+pLtZmAQCAQgAAEI3JoAIvPW5Y0nh+CIcymxtwR76UO2S+SHDwhAAAIQgIAngMj0JB70ifCoV+yz2Uv7vl0vWzxDAAIQgMCTCSAyH1h9Lz7cJz9lCZzJXtr27bLZ4Q0CEIAABCAwEUBlTCwe0/ICxH3yU5bA2ey9/bJZ4Q0CEIAABCAQEkBlhExuv8cLEURm+VLDvjxzPEIAAhCAQB0CiMw63Kt6RejUww/7euzxDAEIQAACZQkgMsvybsIbQqdeGWBfjz2eIQABCECgLAFEZlneTXhD6NQrA+zrscczBCAAAQiUJYDILMu7CW8InXplgH099niGAAQgAIGyBBCZZXk34Q2hU68MsK/HHs8QgAAEIFCWACKzLO8mvCF06pUB9vXY4xkCEIAABMoSQGSW5d2EN4ROvTLAvh57PEMAAhCAQFkCiMyyvJvwhtCpVwbY12OPZwhAAAIQKEsAkVmWdxPeEDr1ygD7euzxDAEIQAACZQkgMsvybsIbQqdeGWBfjz2eIQABCECgLAFEZlneTXhD6NQrA+zrscczBCAAAQiUJYDILMu7CW8InXplgH099niGAAQgAIGyBBCZZXk34Q2hU68MsK/HHs8QgAAEIFCWACKzLO8mvCF06pUB9vXY4xkCEIAABMoSQGSW5d2EN4ROvTLAvh57PEMAAhCAQFkCiMyyvJvwhtCpVwbY12OPZwhAAAIQKEsAkVmWdxPeEDr1ygD7euzxDAEIQAACZQkgMsvybsIbQqdeGWBfjz2eIQABCECgLAFEZlneTXhD6NQrA+zrscczBCAAAQiUJYDILMu7CW8InXplgH099niGAAQgAIGyBBCZZXk34Q2hU68MsK/HHs8QgAAEIFCWACKzLO8mvCF06pUB9vXY4xkCEIAABMoSQGSW5d2EN4ROvTLAvh57PEMAAhCAQFkCiMyyvJvwhtCpVwbY12OPZwhAAAIQKEsAkVmWdxPepNCh/epqMWhiMhAEBCAAAQhA4CQCiMyTwLZstpaowq8WtC3PEWKDAAQgAAEIHCWAyDxK8ILjEXta7NXiccGpQ8gQgAAEIACBZAKIzGRUdIQABCAAAQhAAAIQSCWAyEwlRT8IQAACEIAABCAAgWQCiMxkVHSEAAQgAAEIQAACEEglgMhMJUU/CEAAAhCAAAQgAIFkAojMZFR0hAAEIAABCEAAAhBIJYDITCVFPwhAAAIQgAAEIACBZAKIzGRUdIQABCAAAQhAAAIQSCWAyEwlRT8IQAACEIAABCAAgWQCiMxkVHSEAAQgAAEIQAACEEglgMhMJUU/CEAAAhCAAAQgAIFkAojMZFR0hAAEIAABCEAAAhBIJYDITCVFPwhAAAIQgAAEIACBZAKIzGRUdIQABCAAAQhAAAIQSCWAyEwlRT8IQAACEIAABCAAgWQCiMxkVHSEAAQgAAEIQAACEEglgMhMJUU/CEAAAhCAAAQgAIFkAojMZFR0hAAEIAABCEAAAhBIJYDITCVFPwhAAAIQgAAEIACBZAKIzGRUdIQABCAAAQhAAAIQSCWAyEwlRT8IQAACEIAABCAAgWQCiMxkVHSEAAQgAAEIQAACEEglgMhMJUU/CEAAAhCAAAQgAIFkAojMZFR0hAAEIAABCEAAAhBIJYDITCVFPwhAAAIQgMABAn+fV/f6/B2wwFAIXIsAIvNa9SJaCEAAAhA4QKAXeu9v9++Ajb1Dk0Xmv2/3fiFI93JmXDsEEJnt1IJIIAABCEDgZAKti8x/33f3cgITkXnyTMB8CQKIzBKU8QEBCEAAAk0QaFpk/n16cemeqCff9WyCKkFAIE4AkRnnwt6bEOBCfZNCkgYEMhFYE5n9cX8n8fXp/BuU8XF/3ef16uRrlnPjXfj9Mdl5IactfRfMcAgCVQkgMqvif4bz+MW5TO4pF2r1eIpHVGUKgxcIVCKwdD2y1wvVt7/L+O6+8mXOfp8RokJEqvGIzEoVx21NAojMmvQf4tteaEumbRcN63sQmHLhGO5M8BeglhTbELgHgdnrUf/HNvJa0HWd2jdz19KLStX3x8rsW7seScJb+spxtCHQEgFEZkvVuGkssxf1AvkuX6iHReOtbk30txu618ssNgVixQUEIHA+gdnr0e99yPGPbsZH5tO1QF9PjOjcPH45V+1ruS9HIdAqAURmq5W5UVyzF/Vfjv3x8YJuHj0FXzViLuz+EVRkvDO/eKE2dxlG5P1+/Z7VeIwGBCBwaQKz16PY43CbqXw87try+pQwfvF6ZHxt6WuGsgmBZgggMpspxX0Dmb2oR0Sg6hu7aMuL/Nr4yHFFOWa/7xAKWTWODQhA4LIE1DVGZpH0y+W/7vsefgF1dtRTkITxvW//eF36jrS39I0MZxcEmiCAyGyiDPcOYvmiPj2K6imou4uh2FMXXtX3x9DsU/0t5hWRqRYQO5ZtCEDgkgRmr0fdICDV3cnOXYOmpysu4f497s+3+77NtSth/OL1yNDc0tcMZRMCzRBAZDZTivsGMntR70Xe70uHx8fdbnu6eOsLrRGdm8cbxisiM/GGgzHKJgQg0DKB/pqirjfumuOF5E9ojsena9GY0++OpRaj/ujyeH0982PE5+w1zccn+tKEwAUIIDIvUKSrh7gsMiMXcZlwf9H9XWBdO+c7UOau5+j2t4ggMkciNCAAAQhAAAKbCSAyNyNjwFYCsyIzScyd+Q7UcGfUPhYfvtaIOwdb60x/CEAAAhCAgCSAyJQ0aJ9CYFZkJrzD5AI68x2oPrbxUZn/XjzzQv8pVDAKAQhAAAIQuDcBROa969tEdoOQs+9e+juFy+8w9Qmc+Q6U/wv08R0sBGYTk4YgIAABCEDg8gQQmZcvIQlAAAIQgAAEIACB9gggMturCRFBAAIQgAAEIACByxNAZF6+hCQAAQhAAAIQgAAE2iOAyGyvJkQEAQhAAAIQgAAELk8AkXn5EpIABCAAAQhAAAIQaI8AIrO9mhARBCAAAQhAAAIQuDwBROblS0gCEIAABCAAAQhAoD0CiMz2akJEEIAABCAAAQhA4PIEEJmXLyEJQAACEIAABCAAgfYIIDLbqwkRQQACEIAABCAAgcsTQGRevoQkAAEIQAACEIAABNojgMhsryZEBAEIQAACEIAABC5PAJF5+RKSAAQgAAEIQAACEGiPACKzvZoQEQQgAAEIQAACELg8AUTm5UtIAhCAAAQgAAEIQKA9AojM9mpCRBCAAAQgAAEIQODyBBCZly8hCUAAAhCAAAQgAIH2CPwHeph76K58GckAAAAASUVORK5CYII="}},"execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Stacking is a way to ensemble multiple classifications or regression model. There are many ways to ensemble models, the widely known techniques are Bagging or Boosting. Bagging allows multiple similar models with high variance which are averaged to decrease variance. Boosting builds multiple incremental models to decrease the bias, while keeping variance small.\n\nStacking is a different paradigm. The point of stacking is to explore a space of different models for the same problem. The idea is that you can attack a learning problem with different types of models which are capable to learn some part of the problem, but not the whole space of the problem. So, you can build multiple different learners and you use them to build an intermediate prediction, one prediction for each learned model. Then you add a new model which learns from the intermediate predictions for the same target.\n\nThis final model is said to be stacked on the top of the others, hence the name. Thus, you might improve your overall performance, and often you end up with a model which is better than any individual intermediate model.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Stacking used for the Titanic competition","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Data Preprocessing**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\nimport pandas as pd\n\ndf = pd.read_csv('../input/titanic/train.csv')\ndf= df.drop(['Name', 'PassengerId', 'Ticket','Embarked','Cabin'], axis=1)\ndf['Age'] = df['Age'].fillna(df['Age'].median())\n\nencoded= pd.get_dummies(df[['Sex']], drop_first=True)\ndf = pd.concat([df, encoded], axis=1)\ndf= df.drop(['Sex'], axis=1)\n\n\ny_train = df['Survived']\nX_train = df.drop('Survived', axis=1)\n\ndf1 = pd.read_csv('../input/titanic/test.csv')\ndf1= df1.drop(['Name', 'PassengerId', 'Ticket','Embarked','Cabin'], axis=1)\ndf1['Age'] = df1['Age'].fillna(df['Age'].median())\ndf1['Fare'] = df1['Fare'].fillna(df['Fare'].median())\n\nencoded1 = pd.get_dummies(df1[['Sex']], drop_first=True)\ndf1 = pd.concat([df1, encoded1], axis=1)\ndf1 = df1.drop(['Sex'], axis=1)\n\nX_test = df1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We now have an X_train,y_train and an X_test.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Building the Stacked Ensemble","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"The train data will now be split into two parts, which we will call training and validation.\nThe initial learners(model1,model2,model3) will be trained on the training data and tested on the validation data. The final learner(meta_model) will be trained on the predictions of the initial learners on the validation data and the corresponding target variable values.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#splitting train data in 2 parts\n\nfrom sklearn.model_selection import train_test_split\nxtraining,xvalid,ytraining,yvalid = train_test_split(X_train,y_train,test_size=0.5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The initial learners will now be specified and trained on the training data.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#importing the packages\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.linear_model import LogisticRegression\nimport xgboost as xgb\n\n#specifying the initial learners\nmodel1 = RandomForestClassifier()\nmodel2 = LogisticRegression()\nmodel3 = xgb.XGBClassifier()\n\n#training the initial learners\nmodel1.fit(xtraining,ytraining)\nmodel2.fit(xtraining,ytraining)\nmodel3.fit(xtraining,ytraining)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The initial learners will now be used to make predictions on the validation data (which will be stacked together to form the training data for the final model) and to make predictions on the test data (which will be stacked together to form the test data for the final model).","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#making predictions for the validation data\npreds1 = model1.predict(xvalid)\npreds2 = model2.predict(xvalid)\npreds3 = model3.predict(xvalid)\n\n#making predictions for the test data\ntest_preds1 = model1.predict(X_test)\ntest_preds2 = model2.predict(X_test)\ntest_preds3 = model3.predict(X_test)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"1. The predictions on the validation data will be stacked together to create the train set for the final model.\n2. The predictions on the test data will be stacked together to create the test set for the final model. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#making a new dataset for training our final model by stacking the predictions on the validation data\ntrain_stack = np.column_stack((preds1,preds2,preds3))\n\n#making the final test set for our final model by stacking the predictions on the test data\ntest_stack = np.column_stack((test_preds1,test_preds2,test_preds3))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The final model (meta model) will be created and trained on the predictions of the models on the validation data. We will choose a random forrest classifier for this task.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"final_model = RandomForestClassifier()\n\n#training the final model on the stacked predictions\nfinal_model.fit(train_stack,yvalid)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This final model is now ready to be used to make predictions on the test data for the final results.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"final_predictions = final_model.predict(test_stack)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}