{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## WGAN-gp (Wasserstein GAN gradient penalty)\n●**Reference Paper**
\nI.Gulrajani, F.Ahmed,M.Arjovsky, V.Dumoulin and A.Courville, \"***Improved Training of Wasserstein GANs***\" Proc.NIPS 2017,pp.5769-5779(2017)
\nhttps://arxiv.org/pdf/1704.00028.pdf
\n***GAN often suffers from training instability***. The Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only poor samples or fail to converge. These problems are often due to the use of ***weight clipping in WGAN to enforce a Lipschitz constraint on the critic* ***. So, they propose an alternative to clipping weights:***penalize the norm of gradient of the critic*** with respect to its input. They say this method performs better than standard WGAN and enables stable training.
\n*critic : WGAN calls the discriminator a critic","metadata":{}},{"cell_type":"markdown","source":"### Outline of WGAN-gp","metadata":{}},{"cell_type":"markdown","source":"","metadata":{},"attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAtkAAAFVCAYAAAA61/OgAAAgAElEQVR4Aey9C5xT1bn+/0RR8Aq2XvCCB44TquNY8XakMx6F6qHOcKzgBbWtBfq3M3i0Z+KvB61KrVq0VY410+pfZ+qvQK2VYhXUMqMeFbWdFI/aYjtOkYyFqlW8tFzEiojm93lWsjM7mZ1k576TPOvzCdnZe13e9V1h8ux3v2stXyQSiUBJBERABERABERABERABESgYAR2KlhNqkgEREAEREAEREAEREAERMAQkMjWF0EEREAEREAEREAEREAECkxAIrvAQFWdCIiACIiACIiACIiACEhk6zsgAiIgAiIgAiIgAiIgAgUmIJFdYKCqTgREQAREQAREQAREQAQksvUdEAEREAEREAEREAEREIECE5DILjBQVScCIiACjgR62uDztaHH8WLSyYEONPma0DGQdL6kHwfQ0eRDGw0e6EFPvrYUuk92noWu284537rzLW+3hceFGAv0oK3s36/kjumzCFQfAYns6htT9UgEREAE8iYw0DETgYZudPo70OSfj7X51ljXjt5IL9rr8q3IobyX6y6kbRTshRgLNKOzuwGBmR3I997JYTR0SgREIEZAIltfBREQAREQgSQCPVgQAIJzm5PO62PVEGieiyACWODq0UrV9FodEYGSEpDILiluNSYCIlA7BPhI3gefeTWhIy9XcDR0I1qXDz4Tw5FE0oQlWO1F3+PZ0l1LqoYfBzrmo6txBqaCntMAQggh4PehyYpfMfW1oaOjKda/WBiMQzuJZWIhMCZfEzp6GBYzaHPcXgebYEIcrLxJPK367G5ZB1sG60/B05RJ6pe9bus4yW7Tx6T24m1ZZWibdZxUPp43Cj+BCcc8Xr/TWCBFXyyGps2kPplrdWif14qu+fJmW6j0LgKFJiCRXWiiqk8EREAEjCBsQV8wjEgkgkhkHvoDXTlyoYjym9CNaF1hBPtaEoU2hZR/KWaE2VbsFQ6ir8WHto401xy9mANYsTSExhlTUcdQh3AQjWhEMBxBb0KsRxcC/fNi7XWimX32B9DQPWhDONiIUGBBijj0EALzgcUxe5m3qyVVzDpvWLLgmY5HTzLPbrR2tURjz80IJfcrediG2h0K+OGb6bYvQ8sP9jsNQ8exSO6Lw3cjXZ/89WgMLcUK+81Jcnf1WQREIGcCEtk5o1NBERABEUhBoGc5utCKeXFR2oy5wcYUmTOc7lmAQKgV3Z1W6EbUA4mu5UnitQHj7fHOJhY4gs6prD/FNavKBBPC6A8BDQmVJWSIf2idZqtgwI+5kQjiZgKomzoDjejD2hQirnVeOyyT0+bNiWeKPvtXYGmo0RYK04zOJLsT+hXv7eDBULuBoedy6He2DF1/NwDHPtWNRwNC6A8P9k1HIiAChSMgkV04lqpJBERABAyBgbV9QGM9/DYeURFpO5HNYVJdoAfSLl7r2jGvtQstVuiFPf4g3bVsbBiStxH1CR2siwpms+pHLKzDhDcMKRg7kVQ+VTZGWWTLM12fw/0IJd90JLSdyS6n607nEiq1fUiTty5bhhjyPRvy3TAtp2rTj/pGoC/VXZDNah2KgAhkT0AiO3tmKiECIiACniPQ3GmFaUTDH6Lx29Hwi3TXCteRWAx6S58JLTFhKybUpHAtZFNTafqcjUVu8nqLoRuLlUcERCA1AYns1Gx0RQREQARyIlA3vgEI9SPhKbzxoOZUHZAcN5vWGxsNf4hEutGKLixPiLtOdy1H26xiDOloDCJsX6Yvnz5b9TLsJC+eSX1em/QUwNZO2Q9zYZjVdyO5h+5Dg5JL6rMIiEBmAhLZmRkphwiIgAhkR6B5mhG4LfGwjR60teQ48ZFLrTWGEIivtTaAjvldDLJFPCLavjGLZenAWvShEfVrHTbBsa7Zwz2scmjGtFZbCEE2cbt2wcfJh7n2OW5L7CBbnul4TJ2KGck8m1Ks2JJsRyk+p2OYPBZuvhvpbE77PUhXUNdEQATcEBjmJpPyiIAIiIAIZEOA3tNuwNcCn9HWjQgGW4FANnVYeevQ3hsGmvzw+WLnWrsRsc8wbO5EpJti2srAfFwRJLr5S/v41NesVuzvzdNa0TJ/BQbaOTExKrpbWnzoSm43oVAnult9aPH7Yt1k+91o8LdEJ9Y5Cnp7BemOs+SZgQdS8RzIa53FdB1wd605A8O6oWOR8buRpuWBFUsRapyBxdbs0zR5dUkERCB7Ar4IA+eUREAEREAERCBOgLHB81EfE+nx0zqoIgLR5f/65yWuCFNFHVRXRKDsBCSyyz4EMkAEREAEvEdgoKMJfq6DbfeYe89MWZQrAYbUzK9HuHdwGcVcq1I5ERABZwKKyXbmorMiIAIiUBwCZgc+a+dCp/fYrojFad11rXXti82mN/GwctclldH7BDhHoA/BxRLY3h8rWVjJBOTJruTRk+0iIAIiIAIiIAIiIAKeJCBPtieHRUaJgAiIgAiIgAiIgAhUMgGJ7EoePdkuAiIgAiIgAiIgAiLgSQJaws+TwyKjREAEykFg06ZNOOyww/D3v/+9HM2rTREQAREoCIH6+nq89NJLBalLleROQCI7d3YqKQIiUGUEKLL33ntv/O1vf6uynqk7IiACtUJg/fr1mDx5cq1019P9VLiIp4dHxomACIiACIiACIiACFQiAYnsShw12SwCIiACIiACIiACIuBpAhLZnh4eGScCIiACIiACIiACIlCJBCSyK3HUZLMIiIAIiIAIiIAIiICnCUhke3p4ZJwIiIAIiIAIiIAIiEAlEpDIrsRRk80iIAIiIAIiIAIiIAKeJiCR7enhkXEiIAIiIAIiIAIiIAKVSEAiuxJHTTaLgAiIgAiIgAiIgAh4moBEtqeHR8aJgAiIgAiIgAiIgAhUIgGJ7EocNdksAiIgAiIgAiIgAiLgaQIS2Z4eHhknAiIgAiIgAiIgAiJQiQSGVaLRslkEREAE8iWwadMmrF69OqGaDRs2YNu2bXjqqacSzvPDxIkTMWLEiCHndUIEREAEREAEnAhIZDtR0TkREIGqJ0DB3NzcbER1cmcnT56ccGrs2LFYt25dwjl9EAEREAEREIF0BBQuko6OromACFQtAYrsadOmuerf+eef7yqfMomACIiACIiARUAi2yKhdxEQgZojcOaZZ7rqs9t8ripTJhEQAREQgZogIJFdE8OsToqACDgRoCc7U5w1Q0UYj60kAiIgAiIgAtkQkMjOhpbyioAIVBUBNyEjChWpqiFXZ0RABESgZAQkskuGWg2JgAh4kUCmUJBM173YJ9kkAiIgAiJQfgIS2eUfA1kgAiJQRgLpQkYUKlLGgVHTIiACIlDhBCSyK3wAZb4IiEB+BNKFjChUJD+2Ki0CIiACtUxAIruWR199FwERMARShYSkOi9sIiACIiACIpCJgER2JkK6LgIiUPUEnEJGFCpS9cOuDoqACIhAUQlIZBcVryoXARGoBAJOISMKFamEkZONIiACIuBdAhLZ3h0bWSYCIlBCAsmhIcmfS2iKmhIBERABEagCAhLZVTCI6oIIiED+BOwhIwoVyZ+nahABERCBWicgkV3r3wD1XwREwBCwh4woVERfChEQAREQgXwJSGTnS1DlRUAEqoaAFSJivVdNx9QRERABERCBkhOQyC45cjUoAiLgVQIMGTn88MMxceJEr5oou0RABERABCqEwLAKsVNmioAIVBmBp3/9a7wy8EpCrxiycdCBoxPO5fNh2LBhOOSQQ7Kq4sYbb8T69etTlmG8tpIIiMBQAps2bcKvfvUr7NixY+jFNGcOPPBADB8+PE2Oyrq05557Yt999y2b0a+//jp23nnntH/HsjVOf/eyJRbN74tEIpHciqqUCIiACORO4NZgEOOP/1cM333PeCUfbd+Gze++Hf+c70Hkk4+x+e03860mofxbf3014XO+Hw486GDwZiBTOuH443HUUQ2Zsum6CJSNwOrVq/Hiiy/i2GOPzcqGd955B9u3b8+qjJczf/DBB9i4cWNZTdy8eTNGjhxZMBtee+21gtXlhYo+/elPgzdDxU6Z/7IX2wLVLwIiULME9jngIOy+V+IPwYFj/TXFY+Nbb+Djjz9O2+e3X1+HgVdekchOS0kXvUBg9913z/rpUbZPm7zQT9lQ2QR4E/T+++8XvRMS2UVHrAZEQAREIDUB3mhkSv94bzM+3lhYj3ymNnVdBERABKqVwD777AO+ip0ksotNWPWLgAiIgAjkTGDVqlV49NFHcy7v9YJ8pD9q1KiszaT397TTTsu6nAqIgAiUjoBEdulYqyUREAEREIEsCWzbtg0nnniieWVZtCKyb9myBXxlk/7xj3/gt7/9rUR2NtCUVwTKQEAiuwzQ1aQIiIAIiIAIkMDee+9tXtnQyFaUZ1O38oqACBSOgNbJLhxL1SQCIiACIiACIiACIiAChoBEtr4IIiACIiACIiACIiACIlBgAhLZBQaq6kRABERABERABERABERAIlvfAREQAREQgfwIDPSgZyC/KlRaBERABKqNgER2MUd0oANNviZ06MenmJRVNwnou6bvQbkI8Lvnn4+15Wpf7YqACIiARwlodZFiDkxdO3oj7cVsQXWLQJSAvmv6JoiACIiACIiApwjIk+2p4ZAxIiACIpAHAfNEow0dHU3w+Xzw+drQ46q6AXQ0MX/s1WYr5fSUxDrXQy92ACGEEPD70KTHdq5oK5MIiEBtEJDIdjvOufx4WT9EDBexjvmjZP2Q+WI/Suba4A+c/fcN6EFbPH8TOkx5tz+cbjunfJ4ioO+ap4aj8ozpQqB/HiKRCCKRTjRn7AAFth+Bhu5YmTCCfS3wJf4hcq7F347ecBCNaEQwHEFve51zPp0VAREQgRokIJGd1aBn++OVXHkIgfnAYvPjF0E42IhQwA/fzMRzXS2WiKbAbkFfMBz78ZuH/hZ6jZSqn4C+a9U/xsXrYeu0zNI63nrPAgRCrejutMrUoX1eK9C13KUXPF6TDkRABERABGwEJLJtMNwcZvXj5VBh67x2WL6euqkz0Ahg6Lk+rKX3u2c5utCKeXHvUDPmBllCqRYI6LtWC6NcjD42ot6fZb2N9Ugo4q9HI2J/h7KsStlFQAREQASiBCSys/om5PDjlVC/U3mnc9FCA2v7gKQfP0uYJ1SrD1VIIPX3wl1nnco7nYvWpu+aO6rKJQIiIAIiIAJuCUhkuyWlfCIgAiJQrQRCS7HCvtRouB8hNGC89ditWvutfomACIhAEQlIZBcRbr5V141vAEL9CNsrMj9+9hM6FoH8Cei7lj/Diq2heS6CjSEEFlgrigygY34X0DotOmmybjwaEMLSuAofQMdM29yQ2PX+hD9UFUtDhouACIhAwQhIZBcMZREqap6GVnRhfnxZrB60tXQVoSFVWfME9F2r4a9AHdp7YyuKmJWMYiuNxCdCNqMzHAQ4STt2vX9eN1rjxJoxjfMkW3zuViSJl9OBCIiACFQ3AW1G4+nxbUZnpBttPj98ARraiGCwFTDHnjZcxlUcgey+a2vWrDE9PPzwwyuup1VtcM6bElFoR5By6yyneiPWaiRAc2cEkc6qJqvOiYAIiEDWBHwRLqaqVDkEetrgm1+PcO/gKiWVY7wsrSgCab5rixYtwuzZs0GRfd555+H88883x9n079ZgECdOPQ+77zUym2I1mffVl/vw8cY3MX3amTXX/6eeegrbtm3DiSeeWHN9T9XhLVu2YNmyZQgEvOVxWb16NcLhME477bRUpuu8CNQUAYWLeHm4KXJ8TYhHiyAWK9kwPr4MoJfNL4ZtDz30EL73ve/huuuuwy233IL169cXo5naqzPH7xo92hyLI444wryuvfZaWF7u2oPowR4nbXQV39HRvsGVfcKjB7sgk0RABESgUglIZOc6cqX48WruRDQU0toNMjlWMlfjK7PcwoUL8aMf/Qg/+MEPQDEXDAbR0dGB/v7+yuyQW6sr5LuWLLivvPJK0LOlVEYCJsyDOz+mevUivgx/Ac3kk47p06eD75s2bSpgzapKBERABCqHgER2rmNVoh+vuvbexB/I+GSkXA2vzHJ8ZPzAAw/g8ccfx7vvvms68frrr+O2227DPffcEz9Xmb3LYHUFftcouL///e/jmGOOwbhx43DZZZdJcGcY5mq7vHz5chNStM8++1Sk4L7jjjvwxz/+sdqGRf0RAREoIYGam/jI2L5Vq1aVELGaKgSB3t5e9PX1Dalqx44d5ofwkUcewSGHHDLkuk4Uh0A2ISEM6eFTB77Gjh2LadOmYebMmcUxTLV6kgAFtyW6Of5nnnmm+R6MGjXKk/bSqM7OTowcORJHHXWUZ20shGH8/8mnDvw/ybHh/1ElERCBwhCoOZHNPyiMIVWqLALHHXcctm7d6mj0O++8g5UrV+LPf/6z43WdLDyBDRs25FQp//8tWbLETGQ7rE47neQEscILVaLgrnDkGc1nWBdffOI0YcIECe6MxJRBBNwRqDmRzdUQKMiUKovAfffdh6efftoxLGTixIlob2+XB6aEQ2qtLuK2ydGjRxsvGVcimTRpkinG1UWUaptAqQT3F7/4RfM3gnM46J2+6667MHz48Kzg/+Y3vzFzQD788EMceuihpj6/3x+vg+ElPT3RDX2OP/54c51tMT3//POm7ObNm8FrF1xwAexl45V44ECC2wODIBOqhkDNieyqGbka68i//du/mQmOAwMDCROp6uvrTcyvQkW894VwEtbes7IyLHrikV/hyV8tww87KuPGJJcnHcmC+9Zbby3YjTMF8tq1a/GlL33JiOtcBPa5556Liy++GHyqxqcxJ598Mh577DETTsJQqDvvvBNXXHGFqZ9invHcdA5wSbszzjjDlK2rqzNCnGVffPFF7L///p7+Akpwe3p4ZFwFEJDIroBBkokAYzfprd5pp53AH3D+YPLH7uCDDzY/XsOG6avshe+JhHVxRmHiSadg3z2G419PaipOAwWulXMksonbt5ovZrz22WefjWuuucZqKqv3m2++GV/72tfi5adOnYopU6bg+uuvN0Ka80VOOOEEM9GTFY8fPz4+aZJie8SIEXHPNu2gKKdX2+si2w5JgttOQ8ci4I6AlIk7TsrlAQIU2lwW7tlnn8VNN91kNmKgqFMqLwEJ6+Lz32PPvdDQ0BAPtSl+i/m1wNh7t6mYwtpuA2/Kc00UyrzJt6dTTz0VXFaUieEfX/nKV0w8MwV4Y2NjXHCfdNJJJkSFsc7WtTlz5phz9vqyOWbo3O23344HH3wwm2KOebkYQLYpWXBz74LTTz8922qUXwSqnoBEdtUPcXV1kB5req/5oyeBXb6xJXsKBXuMdfmsUcuVRqBUwtrOJVevMWOw6XVOTmPGjAGvMVFwP/PMM7j33nuxYsUKMD6bsdcPP/yw8VYzrOT+++/Hk08+iUsvvdR4wHkt17hszkP5whe+gHPOOSfZrKw/88lgLitu5bPba9ZGqoAIVCgBiewKHTiZLQLlJECvlTxX5RyBymu7HMK6EJQYv00xTIFMMW2l1157zQhpfuYa/pwMyXAUvhgDzsmWdAawPCc+Mp6br7ffftuEmjCGm7vW5pJYJ+O7rUnEudRhlcnmqYOEtUVN7yLgjoBEtjtOyiUCIiACIpAlgUoV1sndnDVrFhiXfeSRR4Ix1T/5yU9MXDVDNpheffVVs3oIVy2hIOdkRyYK77feegvf/OY3jUeb4SIMz+CLnvBKSAxz4RMrjiVFtpIIiIB7AhLZ7lkppwiIgAiIgAsCfMqxceNGM2HZRfaiZ7HionNtiB5ohoZcffXVJtyDQppe6LPOOstUyetc+YiriNBTTXF99913G2HNMBWuOsKyF154ocnPVU5YxqtJa2V7dWRkV6URkMiutBGTvSIgAiLgcQJemy/x0EMPZU2Mk/vsKRAImMnW9nPWMcM3brvtNuvjkHeKbL7o8aYA92LygrC2ngDkGqvuRa6yqbYJSGTX9vir9yIgAiIgAiUi4EWBzRuidevWFWxN8nxQcn1xpnQ3LFb9vGHhjQFvhrzI1bJT77VNQCK7tsdfvRcBERCBmiLAcI6LLroobZ+5XJ99kmPazBV+kWt4jx07tsJ7IfNFwJsEJLK9OS6ySgREQAREoAgEGCOdS/hIEUxRlYBZgYVhIlzyMN/EpRa5ogsTNwdiGI898QaLG5nxxiK5PcbcP/fcc47X7HXoWASyISCRnQ0t5RUBERABERABEcibAEUtN/DhcodHHXWUEccM+6A4ziVxmUTWx3pHjhxp1ja3T07lLpvcoZNtMdSEN1vc9p5tUpife+65piyFOeuwl7Xbk+5JCOvkCjNKImAR2Mk60LsIiIAIiIAIiIAIlIIARS8F64svvghu1sMXxW8uiaKYIUBcXnHNmjUmTpshP5dccompk9e5SzAFMHfLZAw6vdncOIips7PTiG+eZ3mu/nLVVVc5mkIR3tTU5PjKZ1dRx8Z0suIJyJNd8UOoDoiACIiACIhAZRF44oknMH36dONRpuX0MHMd8VwSPdEU6NwIyAoR4RKJnEjJjYK4MgxFNXfkpLeZSzpSbFuJnm961JmXsfg33HCDeVnX7e/My5VilETADQGJbDeUlEcEREAEREAERKBgBOjFpuC1Jy7dx500s00U2azLEtgsz2OKaas+xuHT282dOCmUKaZvvPFGU+7yyy834SXWhFiK/ba2NlM+2RaFiyQT0ed0BBQuko6OromACIiACIiACBScAAU1Bas9JX+2X0t3TIHNssnhJpzIyO3nmegpf/bZZ00oCT3eVgw3r1F0c9nAN99804SUcEdOinFr3W572woXsdPQcSYC8mRnIqTrIiACIiACIiACBSXAuOZFixaZ+GdLJDNG+rTTTsu6HWslEZa3dtKkiKbwprjm+9FHH20ENL3Us2fPNlvbW+tyc9t75uEundzFk17ucePGGdGevDGOwkWyHp6aLiCRXdPDr86LgAiIgAiIQOkJME66t7fXiF8K68cffxzjx4/PyRCKdIZ+UCwvW7bMiOQ777zTxE4zZITpa1/7mrn+5JNPms8U5HPmzDHHs2bNwimnnIJLL70UY8aMAePFKa6tsiaT/hGBHAhIZOcATUVEQAREQAREQATyI8Al9OhxpqhlXDS9xFwJxE2yPMp8Z6J3mqKYkxcZ0sG67WthczIjJ1qyLSauNGKJaHq7ORGS1wYGBtDS0mLqs8d4m0L6RwSyJCCRnSUwZS8tgfXr15vHdk6t+ny+hNOHH344/vSnPyWc0wcREAEREAHvEqAQtotht5ZaItuen97ndCt/pGuLQpsvJREoJAGJ7ELSVF0FJ8DtfidOnIhVq1ZlrPu8887LmEcZREAEREAEvE3g5z//OZYsWZLWSO3amRaPLnqEgES2RwZCZqQmcOaZZ7oS2eeff37qSnRFBERABESgIggwjIM7MSqJQKUTkMiu9BGsAfspnq+88sq0PWWoCF9KIiACIiAClU2AAlsiu7LHUNZHCWidbH0TPE/AChlJZ6hCRdLR0TUREAEREAEREIFSE5DILjVxtZcTAYaMpEsKFUlHR9dEQAREwJsEuJ05X25WFWG+zZs3e7MjskoEHAhIZDtA0SnvEUgnohUq4r3xkkUiIAIikIkAd1Xk6+abbzabw7jJzy3UqzFxsueECRMqpms7r+vHXtdeAN/7WyrGZhpKu4f/6v8m2Oz7yN2ykQmFXH6QyHYJStnKSyBdyIhCRco7NmpdBERABLIlwB0W6ZnmKiF8WetdZ1uP8peHwCeHHIad172EXXsfLo8BOba68/qXMKJ7Ybz0nje3YZffPBT/XOgDTXwsNFHVVzQCqVYZSeflLpoxqlgERKBkBN577z28/vrrJWuvUA3tt99+ZmOUQtVXLfW8+uqr6OzsNN2h2A6Hw2aHxWz7xxCT5557zpTnGtjJ61yzHb6YuJskd4a0J4r8VNfs+fI5Zv+S282nvmKW3WX109hl9TP4x6xvZ2wmsstwbG86A7s+9Ut8OOXLGfN7NcMu//sotp9wWtHMk8guGlpVXGgCTquMKFSk0JRVnwh4iwCfYnFTqhdeeMFbhrmwZsOGDa5ijV1UNSTLP/3TPw05VyknGFdNcczE7c+5wyK3Wc8mUZife+65Jkabm9BwS3Xu+njLLbeYahYuXGjOUXhv27YNFLvc5fHUU081wtsqe8ABB4AhKBdffDG4K2Qh0k033YS+vj5TL0U+62X92STeALBP7CcT7b799tvjgp3hJVdffbXpP0X8l770JVxzzTUmL8tcdNFFpn2emDp1qrEh04otvo1vY+f1/aYON/98PLYewx+7Bzv/9RV8fPBhboqYEJN/zPw2Riy/AzttfhesY1vLbHyy/yHx8hT7wx/7OXz/2IKPD/pnbJsRwCej9jPXRyy7w7x/fPA/Y9fnHge2b8P2yefgowmnxMvTHor/YeHV+GS/Q7DDP8HxRsAKG2E9kT32BnYdgZ3efj0hL0NJ2CZvJCwb4g25OJDIdgFJWbxBwAoZsW9Mo1ARb4yNrBCBYhHg//tZs2YVq3rVWwYCFL4UvIxB5nsm8edk4iWXXGK2RaeAZagJhfIpp5yCz3/+80ZULlq0yIhOS7xTkD755JNGrK5YscJUuWbNGvPOz5deeml8a3en9rI9xzq5+2RTU9MQD3umunhDQJFMcczt4XlTQvt5Y8Dt3yncae/dd99t8rAt5mdbFOMU5xTeq1evNk1deOGFuP766w3rTG1nc33X0K9MdgraD758hauiw/p+iz1vmImPTvoitjX9O0Z0L8IewW9g63VLQO84Rftu99yE7ZPPNeJ4+IqfYO/2U7Gl4wkjcnd+488Y9tJvjSj+6IQp2Omdv2LP+TOxZUE3Ph5Xb2Ku97pquhHeH55yFob9ZQ12W3g9dtr0Dj6YkeFGbvs2k/ejf5kSF9QMJTH9y1Q2Re8lslOA0WlvEkgOGVGoiDfHSVaJgAiIQG3gghwAACAASURBVLEIUHQ+//zzxrNrxXJTuHMTm2XLlhnhSe/2vffea8TmaaedluClpqinUKXwvOCCC0x+CtpCJqdt393WT9FMoU3PtNU/3ozwyS37bYWfMA9DZmj7M888Ew+5IR/2f8SIESYvxXiqNOylVdjrO4kbue1zzliTnUL3/UsWOBalx5eCeUfD50xcdrLIHrb292Dd26YP9eDvOGZSPCRlx5Gfw6iZn8Uu//uYCT8ZsTSID85tx4f//v9FbWg6w4hsim2rDU62pOimKGei6N71ucfwAUX2+peMgP9Ha/SpxHbATM4c1v/skH6wjd0XfdeEi3z0L18w1+nRprC22t+191fGriGFXZ7QxEeXoJTNGwTsolqhIt4YE1khAiIgAqUkYIVQDB8eFVlW2/TkUngyMWyEwpteXf5WTJkyxUy05DWKUnqZGVJy4oknmhePnRI9yNYqKE7vTzzxhFOxnLzzVkUDAwNgjLklsHmewpo3B2vXrjXvtJ9945MeerF53koU5xTqVr95bK/Lysd3hmu8d90S86Igpjc4/nnaHHvWhGNOeGT4xPv/scCEWFBQJ6SPPsRO7zjPo/io/l/iWSlqd4w/xoR2MMyDHmd6q3dbGoy/mMdeP0NTLIHNij7Zf0y8Pt4Y/KN1vsnPMA+KaIpwt4mhJ8Ofvt9kpy0MXdk+6Ry3xYfkkyd7CBKd8DIBe8iIQkW8PFKyTQREQASKQ8AKL2GIiHXMlnp7e+PeXIpKen/p6X388cfB8BGGWzC+n+KcYSR8sY7u7m4jWHmesc32RMdOc3Oz/VTCMSdUOqVUotYpb/I52kEvdHJibLlVL0U2Y9AZu037KbR5jn1iyAj7SXHNEJmOjg7T/2efHerNNSL3yImmqZ3efs2I3R2xz8nt2z8zhIKClLHU9GYPf+p+2Mvx2P7ZXvaTg+vsH00sNGOrsd15Kb0dE07BJ7vvNVhm16FsrItcoo/hKNh1OHb4jzE3EXxn/LebREFNcU7BT+86bwDcxps71S9PthMVnfM0AWtjGrtX29MGyzgREAEREIGCEaBXl17qJUuWxOukB5uTKRsaGsw5xmfTC03BSs91e3u7EdwUqsFg0MRvMyProTilWGeYRXKywlAYiuL0skI3ksvl85keeYp/yyvPuqzPJ5xwggkZ4Q0DBfdZZ51lbiZ4c8CbDN5U8BrFN6/ddttt5jq9/9ZKK/nYxrIMBaEItTy8jH1miIXb9abtHm6W2fnl3xkxbE1+5GRHxk9bL4rcTw5JEuYpOrH74u8acb/5//8N3r/sRyZcxfcRg0bcJbZlhcDs8txj2N747+4KpsglT3YKMDrtXQIU14sXLzaPwrxrpSwTAREQAREoFgFOeOSEPoZEUETTa3v22Wcb7y7bvPzyy811CksKYYZ10PNLYcqVPriqCcM/KGgpTim+WY8XEj3RjKk++eSTzc3Bli1bjL28GWBf+KIA5+RPrszCmwP2f86cOeamgt71q666ynzmzQP7yvrsXn+nfjLUgq9MiV5su4eXkxjRNc8IbTfld++62qwKwnjs3e7rQGSf/cDJhvSqcxWP4f/zc0T22T+6RGDvw9j9tv/C+1d0ZTIren37toQNchjuMez3T+HjzxzrWJ7CPnlDHd400C7fxnfw0eUu23WsHZDITgGmVk5zdvWzzz5Tcd2dOnUKFi9O/+Xfd98DsMceIz3TNz7+mzgx+ljOM0bJEBHwOAE+9n766cc8YeWee+6FT3/6QE/YMnr06Ip2NHDpPG5Cw3e3ifnpWWaiV5nhD/TYUkj/7Gc/M3HMVl0UzFxdg9dfe+0149FlGSZ6tzlRkMvg0fPLcBCGlhTKK03hPmbMYJywZVO6d/arra0tnuXhhx/GHXfcgZ6eHmMXY8zpmbYSJzNSWHNyJ/vAOGwr1IU3GHV1dSZUxJoEyf4VKlGUfvhvg2E1jI/+cPrFJpbaTRsU0hSxO90+F5xw+N61S+KreXww+xrsds/N2OPWb5gXY8Q/+PLlCUv0pWvjg1nXYPcfX20mUzLfx+OONLbt+tg9jsW49B/jtrk++NZ5i00e3jTs3jXPeMRzWbbP3pAvEolE7Cd0XFsEnnrqKWze/ByOPnq3quv4u+/uwNatH3umX88+uw1tbZdg1KhRnrGpnIbcGgzixKnnYfe9vHMjVE4e6dp+9eU+fLzxTUyfdma6bFV5zUt/o/j3hH9Xyp0+/DACrj7X3v5f5TYloX2KWoYlcDUPJRFwIsCVSzixMlW8tlOZXM5x9RN6yO0TJFPVw7y8caCgZ2IIy94Xn2QmUFqrjqQqm+m8PNmZCNXA9ZEjd8LYsbtWXU+91qcXXyz/j3PVDbI6VBMEqvVvVK6Dt2nTx1iz5v1ci3uuHEM5ODkvXSqkpzldO8W4Ro8zN6dJlfbee++sN6tJVZfORwlY8d1ueFh5Ka4ptkcsv9MU41KD+SaJ7HwJqrwIiIAIiIAIiEDOBBgqwRCHdMlaVSNdHl3zPgFOKozsvrc3Dd3+IUZedILxfjN0xI0XPFNHJLIzEdJ1ERABERABERCBohGwJvMVrYEyV8z4cK9MqiwzCrx37b3lNiFl+5x4yVVJLM92yoxZXNASflnAUlYREAEREAEREAEREIHqJFBIgU1CEtnV+T1Rr0RABERABERABERABMpIQCK7jPDVtAiIgAhUJYHXHgNeq8qeqVMiIAIi4JqARLZrVMooAiIgAiKQkcBrtwGzvp8xmzKIgAiIQLUTkMiu9hFW/0RABERABERABERABEpOQCK75MjVoAiIgAgUkQA9yaeOTHwteSWLBl8BWm3lv2fbbdHU/fnEUBD7OePFvhrAC8CskUBW7WZhorKKgAiIQAUQkMiugEGSiSKQFQG76MmqoDJXPoHHgFlXA9/dDDwRe339OODHt7rsGgX2scC4+2LlfwesOxewC+10NY25FFh0A4DjgEWbgfMPS5db10RABESgqglIZFf18KpzNUmAQueJJ4ExNdn72u70a4dFxXGjDUPTWQD6E73PtssJh6FbgVcuBK6cEjt9GDDrQuDxhxKy6YMIiIAIiEBmAtqMJjMj5RABERCByiAwJuY5Dl0KfPtum83H2Y4zHB52eGKGMfz8gDuRnlhSn0RABESgpgnIk53r8A90oMnngy/2auoYyLUmlROBoQRMyIctLpYxtm7jW+3hItZxKClOl3UltxGym/GYLab384Apf6k9g449SSA2bt/uj4ZrMGTEhG940lgZJQIiIAJVTUAiO6fh7UGbPwAEw4hEIoiEg0DAj7aenCpTIRFIIpBvXG1SdZyEtghJMbrHAt9NOvdtS0RTqJ0LfP13sTLfAr7NyWxKnicQegg47IbEcKHX1mRn9itJXmtTvl7hR9lRVG4REAER0I6PBfkO1LWjNxJBZ3NBalMltU4g37haJ36zLAENwMToAhhyLha3S6GGC22T1qYAnDynVBkE7CKZTysSwkYydKHxMuCwF4CfWSuKvAIsuhs47YvRgmPGR1cO6bVWK3kF+G7SDZiVR5vRZICtyyIgAtVOQJ7snEa4GZ2RbjQE/CZcRKEiOUFUoVQE7HG11lJsXDEi53ScgxfS6VysgVf7geS4XEuY52yDCpaEQONtwGmx5fP43Zn1ALDovqgwdiV6DwO6YiuKmO9ebKWR+ETIKdHwkx8fGwsnOhaYxfrtaQpwGoBvj3S/Kom9uI5FQAREoEoIaOJjLgPJeGwrXKS9LpcaVEYE0hCIhWtYy6BxlRB6JCmYlEQgE4ErNwNXJmVibLbrRKGdJr9Zvcb2ZIT1PmGtRhJrxMkG1+0rowiIgAhUBwGJ7FzGMdyPEFrRLYGdCz2VyUTAiqvtsgmZbONqM7WR7vqh9cArSXG8pWw/nW26JgIiIAIiIAIVQkAiO5eB8tejEQEs7+lEs4nDHkBHkx/98xSXnQtOlXEgYOJqL42GecTjaksUF93I+NtzgSWXxeKyH4vF9V7oYGhln/rWxbPwt7c3JHRi1+EjcNSxJ+AzDZ/F5NPPSLhW0R/M05B0YUfcQEbrq1f0GMt4ERABTxGQyM5lODjRsbsfvhYfuqzyrd0I+7mMX2nDRzZt2oRRo0ZZVug9RmDVqj9jwoQxGDFil8pjYuJqGU87MmY7xc99wKxzo2sVF32TmSnAE/cBpx4L/JgmHAd8/cLYcdSkbdu2mYMRI0bEbKzMtz++8L949+238NnjTjAd4PH6gbUIrfwf8/kzR34WP1i4BAeN+afK7KDdaqcwD/v1Ih6vX78eY8eOLWILHqv67+8BG/4O1FfB98ZjaGWOCFQSAU18zHW0mjujy/dxCT++OptRV1dagU3Tp0+fjmOOOQbBYBD8Iau2tG3bR/jc526Cz9eG3XazhU+k6ejq1a/hyiuXDRHYGzZswbhxV+Gpp9amKZ3+0vLlq3HZZUvTZyrEVca0Wttim90bKXw3A/ad/FK1Y9/x0X5s5Xd1LtaeseFJ4FAkTIbcsGED9tlnH1xwwQVYsmQJLNFtNVFJ7/vufwB+fP8j5rXs17/H7998H8+8/Aa+eN5X8PJLf8B3Am2V1B1P2nrdddfhwAMPxMUXX4ynnnrKkzYW1Kjn1gA33DNY5cOrgN6+wc86EgERqAkCEtlVMMyrV1P4XYZx48ZVrOBev/5voDhOTmvWbAC90j09/4kPPrgt+bLj5+uu+xVuvXVGwjUK7ObmH4Lt5JOmTZuAVavW5V1PPjYUvSx3Czz187Yd/mLLuI3j8m2DicKaAptCu1oEt9W7vfYeieuCnUZoPx/6Ne75sbvvnlVe70MJ8MbszjvvxOTJk2tLcBNF8H5g6wdDoXjsTDU6ajyGWObUGAGJ7CobcCfBvWZN0iS2PPtMMUzRSvHLl9vEvE5CmuUXLQoN8RCzDcvrzLAPNwKZ9VOYM1TESnfe+YzxYOcSOuLUZlvbvw6x1Wqr6O+Mq7WW9XN8t4vjHK1huMrXuY62teNk8jJuQ+tNJbgZzlTJ6T+vuh6M0Q6tfLySu+E524suuBc/FhW1K1cDPH7R4e8URa91vf8viYzodeZrw0bg0ecAeqIZAmJP2z+K1nv/r4HnXo7mtV+3jq26X3sHePXt6GvgDevq4DttLLMQ5w3QEUccgWuvvRaF/t0Y7KiORKB2CCgmu4rHmoLbEt2HH344zjvvPJx//vngcT6J4RKjRu0Ohk4w/f738xJEbXLdFL70IrMMhfPhh4/GypX/Z0g4h1O5xYt/a05fd93DmDnzc5g1K328BL3Y5513fEJVL7+8Affee5GxkeEibhKF+U03PWKEPe2+444v4fzzo3G79GZffPHPjZhnX0qaShVXe/6TwPm59cwS3PRyM02bNg1nnnmmea+0+QOf3v8AE6/NsBGl4hCwBDe93KNHjzbfE/6tmjRpUu4N/vR/gD/EROsRhwLfugs4YyLwH7FNdSh8v7MYOHR/4IBPAd+4DTjhM8D1M4FddwF6uTFTTBAzD8XvXd3AwrnAp/aKCu5v3glQaI/+FPDAr4G/b42WZz329MBvop96XwL22A3Yc7doXfd9O9oWr1Kkz78H4LkyJ4prhvfwVcjfjTJ3S82LQFkIyJNdFuylb9T6w0kvRSE8FfQwr1t3IzZuvDWtwGZPOzufwemnN+BPf7oO69bdgFtvPdcIbMtTzbr+8pe/Y9OmD4znmp/pjT799CPjYR8rV34zo8BmWyw7aVLijxxDRyiM3SbGgTOm+3vfm25CVPg+ceI/x4tTdNNTzraUMhNYvnw5Zs+ebUJKOIdg0aJFqDQP99/efgvvbUmzdnRmDMrhgoAluO0hJXmFMHReBgTOjopjeqMtr3LHA8DkCcAtc4DLZwD3XQO88kbUY23ZSW8zr//oUqAzED376PPR9yUrowKbopt57rkKqD8UoFc7Oc37cvTM+ZOACyYDXzgO2L4DoOi20soXgaaGQdFtnS/ze6F/N8rcHTUvAiUnUHMim55dn89XNa9cJhHZ/3A+8cQTOX3pJk0aj7FjP22801YFFJ2cnEiRak8UpfRmM/SCx5ZgfeSRPkyefIt5MVyEeazPN930qL2KIcf0jH//+48knKdo37TpHxg9eu+E89l+oP2sh+8MMZkz52TTV3s97Ds95NmkZct+ZYRmNXz/GP+fS7IEN2/0+D1Uckdg49//hvNmzKjYv1u8sco2WYJ78eLF2RaN5m88crDc6H2Ao/8Z+NOrUS80BfSY/aPhHgzTYCjHYQdFPcpWqUP3A+oOin6id5vH9Fwz0SNOYc3zDCOxwlE+2hG9nu5flqHAtwQ762TYyhcSn8Clq6Ic1+y/G4Vw1JSjD2pTBEpNoOZE9oQJExJXBbFWB6nQ91weqfLx/cKFC7Fx40aceuqpOX3nKDKT08SJ4xzDQNrbTzUCm6Ea1177cLwYQz8iEa7S0onvfOffQeFufV64cGY8n9MB8yd7p+n9ZnKyzakOnqNQt4Q93x955CVzI0Dv9ezZi00st1NZtuEUr+2U1zo3ffq/G+ZmNZoK/b5Ztq9bt87qlut3hgLMmTMHK1euxJtvvpl32JLrhvPM+MZrr4ITIfkqV9rnU5/GL5Yurdi/XbNmzcoanfV36oorrsi6rClQd3BiuU/tHQ0BoaBmeux54KePDb4okBnKYSX7sXXOemf4yM1LgVPnAl/+XrQO65qbdwpqhohQoNOjveeI6E2Am7IeyCPB7YFBkAkVQUAx2RUxTPkbWei42JEjbT9GMfPo9bW81HaLZ89ehAkTDjHhF07X7XndHjvVM2pU1CaGnYwe7W59bMZU2z3v9ILTI05P+re+dTq+8AWbN8xmHD3d9MorpSdQsBjb9M0U7SrDRN547S/getlKxSfg9Hcql6d1xlLL62yZzRjrSUcDY/aLnjnrpKhH2brOSYnphLWVj+/BB4C3NgIMR7G83Yyp3pg0OdJexn5MrzpjvSmwOcEySy/2229vwdy58xAIzLXXWpZjS3B3dHTgq1/9Kk488cSy2KFGRcCLBGrEk80dGX1o6xkcgp42W8hIUwe4jUzpUw/afE3oSGjc6VxullmeIHqsly1bBnqTyjHxjB7mM8+cEBfglsfZ3it6hu0rgtivuT22Np9xqj9VHfSGX3vtGfFXNNb6ZZOd3mx615kovO1p9erXcfTRh9hP6ThGINljfccdd+Q3ia2MZH944zWm9TPOi8XVltGWam26aH+nGB9trQjCcAyGiHBSIicucpMYhmtYq3kwHycyWiEcmWBTsB924KDA5iokL76SuhTbTE5Tjo960xlqMsn9nBFWs//+e2PBgu8V/MlGthsG8feEvyv8feHvDOde7LKLOwdHMg59FoFqJFATnuyBjpkINHQjYrZABwY6mtDSF0Q40o46RLdE97eNNxvKlHKQe9pa0IVGBBMabUZn93L4ZnZgai/tyy45eYKyq6HwubkqCFfqeOutLWZCI0MyfvvbKxJEdaZVQ9xaRZFMkW2JY7fl7PkovBkqwteJJ47DL37xHLZt22FstvKxjZKvLGI17sH3SvdYJyPlTpCPPng/HvrFz4wX+9yvXpScRZ/zIFCyv1MM5WA8Nr3UXDmE3mOmuTOiq4uc+93ohEUK8KMPi05MdNMvimKuYMIVRZgouo/4p+i7U3muQHJXT3SZv5lTojnoveaKJfSEW95wp7IeO0dhbR8/j5knc0TAUwRqQGT3YEEACIZjChv8HEJrd29MwNahfV4rAi3L0dPZDCvX0FGih3k+6sO9aM9W+SK5LD9TYKdIzXMRnO/Hgp52dKY2KF7Y/gevFJ5qrtaRTagEPcWjR4/Ehg2bjWhlebdx08zH+Gu3ieL60UdfMpMVk8swnIR1ZWqbYS9cYrCz89d48MHVJmSEkx+txAmaDDFhDHotp2oR1gwHOebAPRyH8qjj/gW3/3y5WSvbMYNOuiZQ6r9TuODz0QmKG7cCR4xJXLmDYpsrhvwptgEWvdL2UBGuBMIVQOzp4i8O5jGrhBwfDffYdWfghLOAXYcBr7wZLXHC4dGJlVb562ZGJzcy9tpK9G7zBoAebY8nCWuPD5DM8yyBqhfZAx3z0dU4A2FLGA+sRR8aMcNvG5PmaWhFC5b3dKI5SdTS6+0PhAC0ojvSOyjCBzrQ5A+AV5gag2H0JqnvlGV7lqOrtRuRuWvR5Hfaojsq/H3zOzC3Ob03m5PISp1yCeuwi9Rs7KUgpkh3mzjJ8phjvmvCO5JXGeGNgdu6GPPtFPdNOxivHQicmtWNhlv7KyUfHytz8mKlpy99/T/w3mbnpfkmN5+hWOwCDTAnWpclUUxb3utkA7jKB2OjnZJTmWRvM0Uy1962J6s+XrOHiPD47H+154yKbnrCs4zHTqykeJ8krIvHVjXXDoEqF9kDWLE0hMYZiwfDLsL9CKEB8yzR7TjWg55miudIJDlzD9r8xj2OCIW1Edx+tI2PoLM5U1kAzZ3R0JWBNOss++vRGFqKFQPtOXjOHTtVEycprK+44nQTnpK8tXohADA2mztXZlr9pBBtqY7iE/jy1y8tfiNqQQTsBBj/feVd0Rjxi1oGveP2PGU6prBmjLW1eVSZzFCzIlA1BKpcZIfRHwIa0ivq+GD2rR0A/CuMd3lGOIIh2jqeM+mgrh29kfao2PYtRVZlk6qKf6wbjwaE0B8GBu8Q4ld1kIYAvebBYG7rf6ep1lzi2t4U77ls0Z6pbl0XAREoEIEnFhSooiJUQ682VzbhOt2cgOmh9Pvf/95D1sgUEah8AjWyuoi7gWoYXwcYwTwP/f7o6iNNiUt/xCpqRmekGw0Bv9kcIp7HVVl3tgB+1DcCRvi7LaJ8hgAFMJffK0biBM3k9bmL0Y7qFAERqGICXzjBcwK7immrayJQNgK1J7IZhoE+0GmdOlFER8zySIsxM7bLWhviKwAyPMTXgj4TShJJisXOUDZ1o7oiAiIgAiIgAiIgAiJQJQRqT2TbwzCsQeRERLRiWtKkR16ua++NrUU6DcutNa1NXHcr5iVNdLSqs94dy1oXM77HQl3oXVcSAREQAREQAREQARGoKAJVLrKbMa01OeQieq5rvrUBzQA65ncBrdMGVw5xHEJ6qGPL9xlveBeWD7q2h2x2k1iFrWzihdSfYqug1NtXQUmdW1dEQAREQAREQAREQAQ8RKDKRTbQPK0VoaUrEnZ0bO6MoLshAL+Pcdf+6EY1bhaktgaOsdfdrehqsXaNjNYx1582BsUq7ep9YMVShBpnYKoc2a54KZMIiIAIiIAIiIAIeIlAla8uwuXypqG1Zf6QpfAotCOdeQwFl+HLqwITixJdlWSIGdGlB1vnWRvmDMmgEyIgAiIgAiIgAiIgAh4mUPWebKAZc4NAYEE8tsPDwxEzrWcBAghirkOMuPeNl4UiIAIiIAIiIAIiIAI1ILI5eXExgn0taKsInd2DtpY+BBen3+lRX10REAEREAEREAEREAHvEqj+cBHDvg7tvRHvjkKCZZwkKRd2AhJ9EAEREAEREAEREIEKI1ATnuwKGxOZKwIiIAIiIAIiIAIiUOEEJLIrfABlvgiIgAiIgAiIgAiIgPcISGR7b0xkkQiIgAiIgAiIgAiIQIUTkMiu8AGU+SIgAiIgAiIgAiIgAt4jIJHtvTGRRSIgAiIgAiIgAiIgAhVOQCK7wgdQ5ouACIiACIiACIiACHiPgES298ZEFomACIiACIiACIiACFQ4AYnsCh9AmS8CIiACIiACIiACIuA9AjWyGY33wHvFolGjRuHBB7fi6ae3esUkV3Z8/PFWHHbYga7yeiXT1q3bvWKK7BCBiiEwYsQIPPpo/n+j9t13N+y5Z3X85O3Y8QmGDauOvlTMF1GGikAOBPS/NAdo1VRkwoQJ4KuS0qJFizB79mz8+te/xiGHHFIxpvNHkTc1SiIgAu4JTJw4EXzlm959911s3VpZzoR0fdbfknR0dE0EvEFAItsb4yArsiDw4osvmtz80TzppJOyKKmsIiACtUpg3333BV9KIiACIlAqAorJLhVptVMwAk899ZSp6+mnny5YnapIBERABERABERABApJQCK7kDRVV9EJbNu2DatXrzbtrFq1qujtqQEREAEREAEREAERyIWARHYu1FSmbATswppim6JbSQREQAREQAREQAS8RkAi22sjInvSErBCRZiJAtsuutMW1EUREAEREAEREAERKCEBiewSwlZT+ROwJj1aNVmhI9ZnvYuACIiACIiACIiAFwhIZHthFGSDawJ2TzYLafKja3TKKAIiIAIiIAIiUEICEtklhK2m8iOwZs0abNq0KaEShYsk4NAHERABERABERABjxCQyPbIQMiMzAScBPWGDRuwfv36zIWVQwREQAREQAREQARKSECb0ZQQtprKj0Cq0BCGkMyaNSu/ylVaBETAkwR4I82nWF5Pe+65Z0k3u+GOj9r10evfCtlX6wQksmv9G1BB/U81yTF5MmQFdUmmioAIZCBAgf3KK6/g4IMPzpCzvJc3btyIP/zhDyUxYseOHdi+fTsuvfTSkrSnRkRABHIjIJGdGzeVKjEBxmKnEtnJkyFLbJqaEwERKDIBCuwTTzyxyK1UTvVbtmzBsmXLKsdgWSoCNUpAMdk1OvCV1m2neGyrD9qUxiKhdxEQAREQAREQAa8QqEhP9nPPPY8/9vV5haHn7Rh94IFoOf0LnrcznYEjRozApEmTTBY+Ju3r68Oxxx5rPvOakgiIgAiIgAiIgAh4iUBFiuzX//pXjPj0aOx/yDgvsfSsLb0PL6l4kU2BbYlsriYyefJkrFy50rPMZZgIiIAIiIAIiEBtE6hIkc0h23Pkp7DvQWNqe/TUexEQAREQARGoUQJf/OIXccMNN+Coo46qGgJ33HEHQqEQmpub8aUvfSltv37zm9/gj3/8Iy6++OK0+XSxfAQUk10+9mpZBERABERABEQgRwIUmZs3b86xtPeKUWBfffXV0jKhlQAAIABJREFU2H///TFy5MiMBvb29qKzszNjPmUoHwGJ7PKxV8siIAIiUHACAx1N8Pl88Pma0DFQ8OqdKxzoQU+p2nK2QGcrmMCrr75awdYXzvTXX3/deOVvueUWTJ06tXAVq6ayEajYcJGyEVPDIiACIuBZAj1YEAihtTuCzuYSGTnQgSb/UswIN6NUTZaoZ2qmyASCwSA6OjqMN9rv9xuB+fbbb+Ohhx7KuuUPP/wQ119/PegNZmJ9t99+O44//njzmUKe64rT+01P8UknnYQbb7zRHDMDyz7wwANgPl67/PLLzbspnPTPTTfdlHRm8OMVV1wx+CGLI7K4//77wX4wDOauu+6K2+a2Gpb9+c9/jieffBLDhw9HS0sLzjrrrHhx9m3RokUmxIRceI2cmFiW7TNUhXyamppw6qmnxsvqIDcC8mTnxk2lREAERMB7BAbWog+NqI/+bnrPPlkkAjECFLQU2LfddhvefPNNs2vvihUrcuZz1VVXgeUpTrmB0WmnnYYzzjjDiGZWSoFN4cklXx9++GGEw2GwDBOFKV+0hdcZ433hhRca4elkEMM0Ur2c8rs5R8812z300EONwHcTLpJcL/tLpscddxy46tYll1wC64aAYTWnnHKKuaGhwOYKXSeffLLhwHp4k3HzzTdjzJgxJs9XvvIVLFy4MLkJfc6SgDzZWQKrtux8tOwPhAA0IhjuRXtdmh7GPVYZ8qWpQpfcEdC4uOOkXDYC5v9nAPzfHPL7EGjtRsSVO3sAHU1+mD8DrC65nNP/e+tc9wwsbRlsc2kwjN60f0Rs9uqwpglwMx16jK2wCE7eGxgYiIu+bODQC0tByEmQlueWx2vXrjWik+EXFJn02lK88nX33XeDXnMmXqMopcDl65prrjETD1PZkIunPVVd1nnadsABBxhhTy7ZJt5gPP/88+YmgX1gamxsxDe/+U3D5K233jL9pKednmoyo/fcimln+fb2dsyePduUpSd727Zt2Zqh/EkEJLKTgNTWxywfLde1ozfSXluIytJbjUtZsFd6o/z/GUYsdMPtjXBMYDd0I9LLYI/oZ1+bS4Huz6XNSgct+wtBgKtitLW1JVRF8UcPc7aJYpqJ3mt7oseWHmcmCmd6Z+lBp4idPn16fPUOruKxZMkSTJgwwXiTeY1inZ5vp8SQk1TJSSAzv2UHy1HAOuVLVaeb8y+88ELcE27lJw+KabLmzQyF/JQpU0zbn//85xEIBOJ95HV69mknw0z4OVX/rfr1nplAYUS2zYOS3GRJYwOTG9fn9ARij5Zn6NFyek6lvqpxKTXx2m2vZwECoVZ0G4FNDHVon9eKQMty9HQqxrp2vxjF7zm9yRSAhUjWhmTJopD1W2EXjC/mHguPP/44enp6THjEvffei8cee8zkefrpp024CeOZ77zzTuMB5zUrZtluJ8MqUiUn8cxYaLvIZkhGoRP7mmyr5bVnW2TD/jAshk8R+M7r9MozTIWe/yOPPNJcY5gJX7wx0fKA+Y1UYWKyjYczgkjEeoURbIw+dnT1tDK/Pqh0LgTiN0YhBPw++Np6MtfCMvYVC6zPPTzP1QyiryYuaWCuDZ4brL4HbfG8TegwZdvgovXM9lVDDo1LNYxiZfWhsR4J99n+ejSiD2u1WkhljWOFWUsxSu+rlSgSu7u7rY9ZvTM8giKS4RL29MQTT6ChocGcsiY90kPL2GuGizA/BTBDJbh0Hq8xtOTFF1809VGQOyUK01Qvp/z0lNvzZ1r/2qmOTOcOOeQQ47G253vuuefiYTIMjWE/KZopthm3Tm6M4SZ7etvZ//vuu8/cjDAf47R5TSl3AoUR2Unt97T5EUAQYSnsJDIe+mgeLQfRaGKxIy5jN53sDyEwH1gcu8EKBxsRCvjhm5l4rquFQpoCuwV9wXDshmwe+mPxnE411+Q5jUtNDrs6LQK1RoDxvwzzYIgGxdzRRx8dn6SYLQsKbK4UQu8rxTLFJFfoYLzxBRdcYKqj15bhEPTgUlBSXNLzS6HJF1clYYwyr/3kJz8x8dpOXulsbStV/rPPPtvwI0sKat48sI/0UvPFxPhs9p/J8vBbNyj0zpMdE3nut99+Jg+PlXInUHCRzQlbLV18/NiOdHPocjdZJb1GoHXe4FjXTZ0B8xBjyLk+rO1Yji60Yl58YlQz5ppHHl7rUXXYo3GpjnEsei9CS7HC7rUO9yOEBozXH/Cio6/lBhh/TY8q47ItkUxPajaJItgSi5ywRy8044+5QQuFJb3HFJFMXJJvzpw5JlSEgpIhJlxlhIl5uSoJV9ygx5vhJCxriVOTyeP/kOfPfvYzI6IPP/xwc/NCk9kPJl7npEeKcN7YHHjggYY7b3aYGC5CdjzP8mTEJRCV8iNQmJhsy4aeNvgDQDDcqfVSLSZV/+60XJjTOaC/vw9onJHwaNqI8kB/1VMqfQedxsDpnMal9GPjoRab5yLY6EdgQQ/azZPHAXTM7zIrjMTXvK4bjwaEsHTFANrNDfIAOmZyRZFGzGBXYtf7OV9NwtxDg+ttU+g1fuedd4y4syxlLHQ23mNLQFrlGYaRKhSDYpxC3Fo9wypjvXOio7UyiXWu1O8MY8kmUTTb1+Vm3DnDQOjF5kolyV5oTnTki2Ka1ym8rcQbCsal0wvOlUgq6QbD6oMX3wsnsnva4Gvpy7wMnBcpyCYREAERqEkCdWiPLkkCny8GIHkJPzSjMxxEk98PXyCap7W7G60t82MFmjGtFWhp8aFrSNmahKpOuyBAEXfuueeakA5OBOTEQAq8WbNmmXdrxZBUVZ1wwglDRGSqvLV23vLep+p3OgFN4W0X36nq0Hl3BAojsjlZq6XL7DIWjwRw175y1RCB+voGoKsfCQ4v82i6hiB4sKsaFw8OSq4mmUno0ce/7qug0I4gbSmneiNxXzeaOzmvw32LyikC9Lo+++yzZrWPl156yaxLzbhiCjx6Wu2rcTjRolBM9tQ65avUc4yPJodUif1niIeStwkURGT3LIhtRkBPRkJ/XWxwkpBfH6qawNRpaA20YH7HXDSbu7EetLXwG9Na1d32fOc0Lp4fIhkoAtVIgBMPk5edYz8pINN5W6uRRXKfJKCTiVTm54JMfDRejPjyfdYyfnx3uyFCZcKT1dkSaEZnpBsNXH3ELOM3H/VBCexsKRY+v8al8Ew9UGPSMprWEpuD703gapu1lBirypAEJREQAREoBYGCeLJLYahX23jjtb/gl3f/Xzwf+jX++ML/YtfhI/DZ407A2LrPYM43r8Kn9z/Aq6YDTo+A01mbnD/5M8tmPEdBF0H8yXJPGwLJ6/Sms6EWrjkxTNfv5PzJn4swLosWLcJNN92E8847D+eff76ZjZ7ORF0rAwGn70EZzGCTnOTGNG3aNIwdO7bkVnApNy7pxsfvDEfg5LB0icu4Mf/q1avjq1Oky69rIiACIuBEoCCebKeKU53btm0bNm3alOpyRZ1/dPl9OP+0z2Hhj24xsWHnfPUifLn1EtOHX/70Lkz/12Ow8pHoEkEV1bFiGcvJsfbNbLiFM1cyaBjvelGC119/HVxgn/F8q1atwtatW4tlbe3Um+O4UKhcd911OOKII8zr2muvzSheageqemonwM09LrvsMowbNw7HHHMM+F2hgC1VorjmizHAmQR2qWxSOyIgAtVPoOQie8OGDdhnn30wffp00BtWqYL75Zf+gGsCc7DXyFH46a9W4sf3P4Krb+rAf151vTnmuV1HjMANV7Rj+4fbvP9NKsWj5eZOhIOI7jBpwkX8CDR0u94IJxQKmcX1f/jDH5qlhr7//e/j1ltvrdjvkKsvRQWMC/shwe1qNJUJMOKaN2cU2xTdFN9PPfVU0dgwRIRbZdODzaXJ8gkXscR6KmPpAWd7TonnU11zyl+Oc/ybqpuQcpBXm9VKoOQi2wK5fPlys15lpQpuSzz/YOESHHXcv1jdir/z3OxL/w/+9vZbuO+nd8XPe/bAPFq2x9MnHxcmvr6uvTe222Osfpe7gvIP/09/+lP893//t9mRa2BgAA8++CB+8IMf4Ec/+lH1erQ9Pi5O32cJbicqOudEYP369SaUZPLkyWYTDK5hzN+GQiYKX+4AyJARbrBBoZxtYpkTTzwRU6ZMMWEk3KyDW3Zbibvo8YaBG5lwow/ms8Q8hTXL8jxfzEd7vJiuvPJK81SKNnJb7UceecSLZsomEagYAmUT2XZClSa4GYfN+OvGyf+Gzxz5WXtXEo7P/epFxsv95a9fmnBeH7In8MwzzxjvdXJJPgnheqoU3UreIyDB7b0x8apFfMrJp5t8yrnbbruZ7bD5mSGG+SRuTsJd7bj5BjcvYahZNunDDz/ERRddhNNOOw1vvvmm8fSyzq985StxIc3tqrnbIENg1q1bZ85zYxUmbm3NdYv//ve/mxc3PGF+LybugsjEmx/a39zcbJ48c2vySn7y7EXWsqk2CHhu4iMFtyW6OUnmzDPPNJNlRo0a5ZkR4SRHpnF1n0lrEydBOnm50xYqwkX+sWQMZCUn7lLFH2GnRIFND4y8Lk50vHPOEtwMFZg4cSLOPucc7xhX5ZbkGo7Bm9hcY6f/8pe/GLFGtBz7bBKF9ZIlS8xr0qRJCbvaZVNPIfLSix0Oh8EbfSauzcxd9u644w7cf//9xuNLcbps2TKzHB2XnrMz4zWWp+ebuxlSjPPlxTR69Oj4mFn28TtgjQWfNPD/Lic8n3766ZrwbEHSuwikIOA5kW23M5XgtucpxzFDQJjG1vmHNE8PNz0f9rTXyJFpPd72vMU4njNnDr7zne8Uo+qS1dnV1QX+ADjF8NfV1RnPUE9PT8nsqfWG6NVKtT1xKja8UbbfON8aW3EiVX6dHyTw/tb38PWvfBnvv//+4MksjihUc0kcs6OPPjqXojjllFMwc+ZMU5Yr0WR7E8y2uXINQy3KmfikjJ5o+8YnPKZn23qCdtddd5m/Qewz89Jbfs0114BbeV9++eXmaRt3N2S5qVOngn+Tjz/++HJ2K+e2OeGcL8bSM2yGYpvOsFy/YzkbooIiUAEEhvHRHMWs15MluOmxvOY71+LgfQ4sm8m7jhhu2v7QYULjty6eBYaT2NPxjf9qJkPaz+k4OwInn3wyfve73w3xiPGHePz48aDQVvIegWRh7T0LK8OiPfbcC3f/7B5Mn3ZmZRicZOUvfvGLpDPOH+n1pbC2nmAyF73w+YaMOLfm7iyFsjVp0b5dNT3cDP1goqim95rn6LHm00N6rxmewvL33XefCSF5/PHHTdjFGWecYVY6sdfnzprscmV6gsGng/anDE5OjHQtsixfXKKR/9fp4c42HCdd/bomApVOYBgfcZUyMdaLkyqySfRg0gPG/8C8W162/MFsihc87777Rde+fiH0GyTHW3//jkUJnuyvn316wduvxQrpMfnqV7+KXXbZxXhR3n33XRxyyCFmgtE3vvEN7LnnnrWIxZN9tv6/UijRy6UkAoyHTpesv+98t+KC0+Uv5TWGf9ADzQmUjMVmev75543wbmpqMn/vOdHxtttui+9UyPycZMnE7bGZuIMfy9OTzd9ACvJcRfaWLVtw1VVXGW+yqTzFP5m8y/y/+pnPpA97TFF1/DTr4I1RW1ubuRnizYWSCIhAlIBnw0WsH2pLWHtpwBo/P8VsOsNl/JKTF2Kwk22qls+NjY1mGS56td944w3zqJI/chLY5R9hL/9/LT8dWeBEgAKQ4SQU1vSCejVRCHNOCFcO4VKA9ExzNREKZstry1jrCy+80Hi2KbAXLlxoQkLYJ/7d4sRJeuMbGhrQ3d1txDXDTXJNe++9N2688UZjV651OJV79NFHjRPD6Zr9nPXEwYrNtq7ZY9Gtc3oXgVom4CmRzZ3ALI8GJ1d4Ne2190hw5ZB7fnwbvhNow3XB+P6FXjW5auxiWIhCQ7wxnBLW3hiHSrKCS9hRWNPzye9PqRK90fS0uk0U1pzcSEHNxGPGUPf29pqlABmDTY+0lRh/TTHNtfw5J4cC2AolYb7bb7/diGs+OaZjgJ8pxr2WMj1F4O8zn1Bx/DLl9VrfZI8IlINA2UW2Jaz5h5d/gCsl/df1N+G9LZvw0C9+Zv6ofvmi/4ivJPLels0IPfkYfnhjdLLhXnt710tTKbxlp7cI8Md21qxZ3jJK1niWAMOGKMwY9lWORJHNl9tkiWx7fnqtLc+1/TyPKZgppu3C256HgtsS3fbzlXDM32XeoPD/fClvjCqBjWwUgUwEyiKyK1VYJ8O0PNgU2txinUv27bv/AfGJj/R4cwfI2d/w5pqoyf3RZxFwS8DLj/fd9kH5SkfAS08mGVtNj3S6RM91LSf+RvOmiM6vct0Y1TJ/9b16CJRcZPM/Lxfrr5ZEoT370m9iZc/DeLn/j/jb2xtwxowvY1zdeBzfeDI+vX/6CT/VwkH9EAEREIFKIJDrZMNK6Fu+NjIUhMu9ZpowmW87Ki8CtUKg5CK7GsGOrRsvb3U1Dqz6JAIiUHUEKLIltJ2HlRM8lURABApHwBPbqheuO6pJBERABERABERABERABMpPQCK7/GMgC0RABERABERABERABKqMgER2lQ2ouiMCIiACIiACIiACIlB+AhLZ5R8DWSACIiACIiACIiACIlBlBCSyq2xA1R0REAEREAEREAEREIHyE5DILv8YyAIREAEREAEREAEREIEqIyCRXWUDqu6IgAiIgAiIgAiIgAiUn4BEdvnHQBaIgAiIgAiIgAiIgAhUGQGJ7CobUHVHBERABERABERABESg/AQksss/BrJABERABERABERABESgyghIZFfZgKo7IiACIiACIiACIiAC5ScgkV3+MZAFIiACIiACIiACIiACVUZAIrvKBlTdEQEREAEREAEREAERKD8Biezyj4EsEAEREAEREAEREAERqDICEtlVNqDqjgiIgAiIgAiIgAiIQPkJDCu/CbJABESgVgn8+Y+/w7Bdd03b/U8dcBB22rkwf6r2PWhM2rZ0UQREID8C77zzDp599tn8KklTetddd8V+++2XJof3Lw0bNgyjR4/2vqGyMG8ChfnlytuM7CoYMWIE1rz4v3gj/FJ2BSso9/Dd98Bue+9TQRbLVBHIjkDz6adjw4YNGQu9+vLvseOjHRnzucnQ+/ASN9ky5tl52DDse8BBGfO5ybDL8OHYY5/0omHL397BfiP3cFNd1eWhIPnTn/6EN954o+x923333TFq1Kiy27F9+/ay2+BkwOGHH45NmzY5XSrYuQ8//BAvvPBCweorR0U7duzAX//616I1zfrXr1+Purq6orXB/5cHHnhg0eovVcUHHVSYv+Op7PVFIpFIqotePc8v0Ouvv+458375y1/inHPOKYhdW7duxbvvvluQunjHzD9+SiIgAoUhUMi/Qdu2bXN1s8H/w7Xo/Sok63xHv5B/l/O1RX/X8yVYveUpsCdPnox169YVrZNe+n+ZTyfJqpipIkV2MYHkU/cRRxyBhQsXYuLEiflUo7IiIAIiIAIiIAIikBOBUojsnAyrwUKa+FigQV+zZg34evDBBwtUo6oRAREQAREQAREQARGoVAIS2QUauSVLorGe1nuBqlU1IiACIiACIiACIiACFUhAIrtAg/aLX/zC1MTHNKtWrSpQrapGBERABERABERABESgEglIZBdg1KxQEasqhYxYJPQuAiIgAiIgAiIgArVJQCK7AOO+fPnyhFoUMpKAQx9EQAREQAREQAREoOYISGQXYMiTPdcKGSkAVFUhAiIgAiIgAiIgAhVMQCI7z8FLJaiThXeezai4CIiACIiACIiACIhABRGQyM5zsFKFhqQ6n2dzKi4CIiACIiACIiACIlABBCSy8xykVB7rVB7uPJtTcREQAREQAREQAREQgQogMKwCbPS0id/73vfi9l1wwQW49dZb41sfjx07Nn5NByIgAiIgAiIgAiIgArVDQCI7z7GeNGlSvIYRI0aYLdUlruNIdCACIiACIiACIiACNUlA4SLVNuwDHWjyNaFjoJwdG0BHkw9tPTEbBnrQk489nuhTGXjmy82Y3IO2sn8fysBOTYqACIiACIhAmQlIZJd5AKqx+YGOmQg0dKOzGQAFsn8+1ubT0bp29EZ60V6XTyUVVrYQ3EyXm9HZ3YDAzA7kc59TYfRkrgiIgAiIgAiUnYBEdtmHoNoM6MGCABCcS4Wt5AkCzXMRRAALrCcLnjBKRoiACIiACIhAdROQyPbk+PIRvw8+82pCRw9DQNqQu0aKhm9E6/PBF4/jiHXehGNY7UXf41nSXXNgN9AxH12NMzCVXmfjjQ0ghBACfh+aGMNi6mtDR0dTrH+xfjm0Y/KzDXMtFgJjHRsmgzbH7XWwCXDDM4mRnberNjOVz6LPbM+fxM30K6mN5E4bOx3aQR3a57Wia7682Y5fD50UAREQAREQgSIQkMguAtT8qqQgbEFfMIxIJIJIZB76Wyi4ck0UZn4TvhGtL4xgX8ug0DaCbilmhNlW7BUOoq/Fh7YOir0U1xwV/wBWLA2hccZUmMgOhnmEg2hEI4LhCHrj8R5dCPTPi7XXiWaKYH8ADd2DNoSDjQgFFqS4sQghMB9YHLOXebtaUt2EuOGZzCiCcLAPLU12UZquTTfls+izI7fkNpLGMf71SG4ndsFfj8bQUqxQzEiclA5EQAREQAREoJgEJLKLSTeXunuWowutmBcXpM2YG2zMpaZomZ4FCIRa0W0CpHkq6tVE13KbgG3AeHu8s4mBjqBzKvOnuOYYDRJGfwhoSKjM2fTWabYKBvyYG4lEY7hj2eumzkAj+rA2hShsndceFfLsUbq8bngOYQTUtc9DaygxxCJlm27L59PnIW04jWMUXgJbC3/deDQghP6wdULvIiACIiACIiACxSQgkV1MujnUPbC2D2ish99WNioibSeyPUyqD/RqWgK2rh3zWrvQYoWn2EMQ0l3L1oaE/I2oT+hgXVQw97TFQkh88JlwiYRCtg9J5W1Xkg/d8DR5YGNgWLSgC0BfXOWnbjOn8nXZ9hlDvhcJ4xjveCo7/ahvtPcnXkAHIiACIiACIiACRSAgkV0EqJVWZXOnFabRjdauloRY6XTXCtfPWMx0S58JKzFhKybMpHAtZKypMYiwFS5jex8McclQQ9blPdDnDF3SZREQAREQAREQgdwJSGTnzq4oJevGNwChfiQ81Q/35xGTDSA5FtfUlxQGYnrTjE4jMLvRii4sT4i7TnctTxQM6TAi1bZMX759jpnkhmc0T+7xyjmVz6XPrsfRaTzch/I4ldY5ERABERABERCB7AhIZGfHq/i5m6cZgTs/vptMD9paGLiQY+LybY0hBOLrtw2gY34X0DoNJirahGgkTRocWIs+NKJ+LcM3Ulyzh3vETWvGtNakkAS3scB2AcnJmPn0OW4PADc8LUb2taQNF5eb+uRaPl2fk7lZbaQaR3ufnY6tMXUcN6cCOicCIiACIiACIpAPAYnsfOgVpSw9xt1oCPhjYRvzUR9szaOlOrT3xlaiMLHGsZVGrImQzZ2IdGMwJpt5zIoivWhvT3PNPlHSZl3ztFaElq6wbXwSFd5dLQ5LB1rlmjvR3Rpd5s8sM2japze9EBP13PCMMUIAfis23YSu2Dzrlq2O7zmUz9jnZG4ZxtHRrsGTAyuWImQtrTh4WkciIAIiIAIiIAJFIuCLMABWqSAExo0bh5UrV2Ls2LEFqS9eCb2q8+sR7h1cUSN+zXMHjDWej/qwW4Fahg5UFM9C8Iku/9c/L3EFl0LUrDpEQAREQAS8RWD9+vWYPHky1q1b5y3DatAaebK9NuhDwhRi4R0N4+NL1nnN5ER7uOQgbOEpiVdL/qnieRaAGJf/QxDahLMALFWFCIiACIiACLgkIJHtElTJsjV3IkyR6rd2M7SFd5gd/azzTu8uY4iL3Jm69sVmwxv7aoBFbjJ19el4pi5VRVcY09+H4OJKeApSRdjVFREQAREQgZonoHCRAn4FihYuUkAbVZUIiIAIiIAIiED1ElC4iHfGVp5s74yFLBEBERABERABERABEagSAsOqpB+e6MaOHTuwatUq8C5SSQREQAREQAREQARKTWDDhg2gHlEqPwGJ7AKOwec+9znccccd2GknPSAoIFZVJQIiIAIiIAIi4JLAJ598AuoRpfITUEx2+cdAFoiACIiACIiACIiACFQZAblcq2xA1R0REAEREAEREAEREIHyE5DILv8YyAIREAEREAEREAEREIEqIyCRXWUDqu6IgAiIgAiIgAiIgAiUn4BEdvnHQBaIgAiIgAiIgAiIgAhUGQGJ7CobUHVHBERABERABERABESg/AQksss/BrJABERABERABERABESgyghIZFfZgKo7IiACIiACIiACIiAC5ScgkV3+MZAFIiACIiACIiACIiACVUZAIrvKBlTdEQEREAEREAEREAERKD8Biezyj4EsEAEREAEREAEREAERqDICEtlVNqDqjgiIgAiIgAiIgAiIQPkJSGSXfwxkgQiIgAiIgAiIgAiIQJURkMiusgFVd0RABERABERABERABMpPQCK7/GMgC0RABERABERABERABKqMgER2lQ2ouiMCIiACIiACIiACIlB+AhLZ5R8DWSACIiACIiACIiACIlBlBLwrsgc60ORrQsdAlRFP7k66fqa7llyPPouACIiACIiACIiACHiGwDDPWJJsSF07eiPtyWdr67MY1NZ4q7ciIAIiIAIiIAJVQ8C7nuyqQayOiIAIiIAIiIAIiIAI1BqB0ohsE/bQho6OJvh8Pvh8bejJRNoeKmEd9zCEhOWjrybGkphrg+faEiruQVs8fxM6TPlUbQ+go8kHU6eTbSn7EC1n2eTYtyQbmTdlO/a2rX4zZMY6zpZBxrYzMXLRP7vNOhYBERABERABERABEUBpRLYB3YVA/zxEIhFEIp1ozhr5mpWmAAAM40lEQVR+CIH5wGJTPoJwsBGhgB++mYnnulosEU3x2IK+YDjW5jz0twQQStluHdp7I+htr0uZA0juAwWoH4GG7lgbtKsPLU0dGAwl70GbP4CGbvY7+oraviDzjcYQS3JgkLbtTIzc9G+IkTohAiIgAiIgAiIgAjVPoIQiG2idlr20to9Q67x2WBK4buoMNAIYeq4Pa6lwe5ajC62YFxfNzZgbZIn8UkIfehYgEGpFd+dgv+ra56E1FMACy6M+4MfcSAS2LIjaHrMzS3OG9jcNg0xtZ2Lkpn9Z2q/sIiACIiACIiACIlALBEooshtR788HqVN5p3PRNgbW9gGN9bA3aQnz3K1IbM+0gS60xENSGLbSgi4AfUbpA6iri94Y9LTFw1x8/nQe9XTWJbYfzel0LlZHhrYzMXLVv3Tm6poIiIAIiIAIiIAI1CiBEorsKiXcGEQ4FgZihYPwfTDsJBbz3NKHYDgWMhIOGi988YkUoO2M/St+L9SCCIiACIiACIiACFQagaoV2XXjG4BQP8L2EQn3p4nJtmd0dxxtYylWDAZgDy3IkAwjVHsRj1wpsB1DG42dydB2Jkau+peycV0QAREQAREQAREQgdolULUiG83T0IouzI/vZtODthYGchQwNc9FsDGEwEzbREcTFpK0iU7IJsS52keh7UjXpXRtZ2Lktn/p2tc1ERABERABERABEahBAtUrstGMzkg3GrgCiYmZno/6YGuaIY4uVedqab14LVyRJIwgAvBbcdkmLMTmtW7uRHdrCAF/bJlB/1LMCHejFSH0J7jZ45UW7iBj25kYuehf4axVTSIgAiIgAiIgAiJQNQR8EQYQ10qil3l+PcK9g6uU1ErXXfdTjFyjUkYREAEREAEREAERSEWgej3ZQ8I2BtAxvwtoGB9fBjAVlJo5L0Y1M9TqqAiIgAiIgAiIQGkJlEdkO+xCOLhjIsMqkmKac2HS3IlwEINhGr7YpjH2BatzqbeayohRNY2m+iICIiACIiACIuAhArUVLuIh8DJFBERABERABERABESgegmUx5NdvTzVMxEQAREQAREQAREQARFAyUT2QEdTbJUPF6EgJpzERb4SD2BZ++BRJiUeAjUnAiIgAiIgAiIgAhVBoEThItx5sAXojqByQ6Kz7ANFsVmuz7acX0V8JWSkCIiACIiACIiACIhAvgRK48keWIs+NKLen6+5ZSxfDX0oIz41LQIiIAIiIAIiIAK1RKD4Itt4dAMIIbYhS1tPZr7JoRHW554ONFmbvvh8MBvHmGuxjV58PgxWT8+zdb4JHaZsG5xbz7ARTao+JLXNFVJSbmZjlsvzwTdoIIBou4Mrq6Syj1nZ91gIjXWcFQ+rDotJ9D3R3kzMsrA38ygrhwiIgAiIgAiIgAhULYHii+y6dvSGg2hEI4LhCCI5x4uEEJgPLI5EwP1zwsFGhLib48zEc10tFKrR0I6+YNjkjUTmob+FQj9V4s6GEfS21zlncOxDD9r8ATR0R+0ZtGnBUCFPUdzShVbmjfefgjW2rGC8T31oabJt0e5sTexsNjxYJJO9mZjla2/azuiiCIiACIiACIiACFQVgeKL7ALiap03uFNj3dQZaAQw9Fwf1nYsRxdaMS8umpsxN8jcBUwDfsyNJMaYR23qw9oBWzthxmYHgGA4MR69ZwECoVZ0x0U3UNc+D62hABY4u9ttlUYPh/Y9BQ/ak8nengzMCmDvkA7ohAiIgAiIgAiIgAhUKYEKEtlOMd1O54D+/j6gsR72EHBLlBdsHOvqojtHWmEgDE3xJ3vLQwjEPOgN4xO95ANr+wB0oSUe0sLwjRZ0AehLUOmpLHbqu9O5WPkM9hp70jDL395U/dB5ERABERABERABEag+AhUksr0GPxa/3NIXDYNhyIcJi0m0kyEi3a1A13yHMJDGIMKxUBGGm1ivlGEriVVn+cmdvWkrLam9aS3RRREQAREQAREQARHwNIGqFNn19Q1AqB9hO/pwf5qYbHtGl8cMrzCi07ZE35A2op7l5s5uEwYys2MwjqRuPG1cihWDp1w2nGO2DPZG7UnNrOT25thNFRMBERABERABERABLxCoSpGNqdPQii7Mj4vaHrS1MBCjwMkukmOTG51biMaEhwIzETepeS6CjSEEZto83Cb0pIib8KSztzkDs3LY6wxTZ0VABERABERABETA8wSqU2SjGZ2RbjRw9RET8zwf9cHWNIMRXZoucTm7NNl5qbkT3a2xZQlNPPZSzAh3oxUh9P+/9u4eOXElCsPw572AA5dXIK8AnDhyejM5NMnNJmQBIjTZTYluAqzAWoHLgcVeNKU/kIRQCxCNjF5XzYzQT59zHhEcq3pahUfoyTiD9/9KTXW0okkgTxMNs3nZ8dST3JNxQwpHHTbmazKznO9RxXEyAggggAACCCDQLQFLb3zsQNHRU+Lpg4LP3QolHciq2ylg1u37Q3YIIIAAAggg0FmB23ySvTftYqPZdC493icrgnT2dlwxMcyuiE9oBBBAAAEEELg1AftNdvy2wuJbB3dvPIz2tzAnefShwJMmwyxO+tKX3JrUt3Yjz64Hs7MJGQABBBBAAAEEEMgE+jNdJKuYfxFAAAEEEEAAAQQQuLCA/SfZFy6I4RFAAAEEEEAAAQQQuLYATfa17wDxEUAAAQQQQAABBG5O4Hc02fE87hbmat/c7bNQUPwfIt+0rgrFfalSYR8CCCCAAAIIICDmZPMlqBeImuyxtAo/NKo/k6MIIIAAAggggAACqcDveJLN7UIAAQQQQAABBBBA4BcJ2GuyK5bua/yGxfy0hGx7PdNT9qbEuzvFY5VivBXmOKz1tj3/SbP4+gPTINTwDZCleNFShElN1ddvZk+6u8tiJufsli/M9kuKx33TLD4/WoYwPXYwXvaNa1JjTdx4mNIYP9nYFf/G+aTTeLLtY+/L2TWZ6qnIm10IIIAAAggggMClBUIrP6vQlUJ3tQsWeE4ouWFu1+5geSvwQkdO6AVhGMbbCuV4YfQx+knGqtqXjZ/Ed+IBoiuSz43jJ2FKf9fXFOeUyzEMg9BzFCY5JNvKgRTOz2rMHc9yzu8qGjap0RA3dWns1NJ9Ob0mUz2lW8ZHBBBAAAEEEEDAkoCsxAmCbUO8jZdv0LY7D2zkz00b0HxjljXe+/vSxnzl7jX0xQb1QNy63aaa8jlH4+Q/V+RTaKIrazQYVoy5V2PFOYW4Fcf3xsib5GuqzDn65aj4y1XBwWRoyqfieKGefK5sI4AAAggggAACFgXsTBcZDJLXmccrVaRvYRxO5J/8mN7Rw7B8cdW+5JzNz5fkPCh/yeD5VU55iGM+m2oaPOvV8bVYbuJRN8uFfOdVzwMpzkdzjbfTVyKTseaSvn6S86VSPYZ4TWo0xW0yRj1RKef45Kp96Shn1mSqpz5XjiKAAAIIIIAAApcTsNNkK53nO/6SF4TR03OFgXdek3s5k4Yjm2oa6P2PK3+x1EYbLRe+nNfn5JeNKILjKYgcSn8+3wcH4pviHbisvPvouOUB2vzcQk2dqqdNG8ZCAAEEEEAAgd8sYKfJXv+vedwMfWrbQwbfZzzJPo58cP8o+d8K8pedG79JTaMXuf5Cy/VSC9/Vn7T4JJ+F0ofc+awObxviNanRFLfJGIcTPOHImTWZ6jkhIy5BAAEEEEAAAQRaEbDTZEepRs1mNhMiWlFiHE2OsPQTNbuaazrLEljrrY34xppGenF9TcYT+e7Lbp3p0b/yHF+Tf2bKMlI8lcbwwp26eE1qNMVNxxhvl2VpyanuNp9Tk6meurgcQwABBBBAAAEELihgp8kefWgVNZvDbD72Qq/BSq58fRceL1+q0pE+wpUeJ0MlS+ZN9eC5NcGql+ArXNCwptFLEsd9yb/KZaD3z0CeJhpm87LjqTS5J/2FYJKM8ZrUaIqbjOHOxw2dykke+fnsmkz1HJkPpyOAAAIIIIAAAi0J9PeNj9GT4+mDgs/33TzpllA7M8wt1niLNXXmC0MiCCCAAAIIINCWgJ0n2W1le+o4e1MxNppN59Lj/e002LdY4y3WdOp3mOsQQAABBBBA4FcJXP9JdjQ/u3Y5P0deUDONoiF39LbF4SS3aKC7UviRn8LRcKAOn3aLNd5iTR3+CpEaAggggAACCLQkcP0mu6VCGAYBBBBAAAEEEEAAga4I9GO6SFe0yQMBBBBAAAEEEECgFwI02b24zRSJAAIIIIAAAgggYFOAJtumNrEQQAABBBBAAAEEeiFAk92L20yRCCCAAAIIIIAAAjYFaLJtahMLAQQQQAABBBBAoBcCNNm9uM0UiQACCCCAAAIIIGBTgCbbpjaxEEAAAQQQQAABBHohQJPdi9tMkQgggAACCCCAAAI2BWiybWoTCwEEEEAAAQQQQKAXAn8BZrpqJJrmujEAAAAASUVORK5CYII="}}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport os\nimport sys\nfrom tqdm import tqdm, tqdm_notebook\nimport glob\nimport shutil\nimport time # time.perf_counter()\nimport random\n\nimport matplotlib.pyplot as plt\nimport cv2\n\nimport xml.etree.ElementTree as ET\n\nfrom keras.layers import Input, Dense, Reshape, Flatten\nfrom keras.layers import BatchNormalization, Activation\nfrom keras.layers.advanced_activations import LeakyReLU\nfrom keras.layers.convolutional import UpSampling2D, Conv2D\nfrom keras.models import Model\nfrom keras import backend as K\nfrom keras.optimizers import Adam\nfrom keras.preprocessing.image import image","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-01-13T10:36:27.298452Z","iopub.execute_input":"2023-01-13T10:36:27.298756Z","iopub.status.idle":"2023-01-13T10:36:27.305910Z","shell.execute_reply.started":"2023-01-13T10:36:27.298706Z","shell.execute_reply":"2023-01-13T10:36:27.304898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# import glob\n\n# path = '../input/chuviettay/CHUHOA'\n# subfolders = glob.glob(path + '/*')\n# print(subfolders[0])","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.307629Z","iopub.execute_input":"2023-01-13T10:36:27.308216Z","iopub.status.idle":"2023-01-13T10:36:27.319699Z","shell.execute_reply.started":"2023-01-13T10:36:27.308112Z","shell.execute_reply":"2023-01-13T10:36:27.318867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# subfolders= ['../input/chuviettay/CHUHOA/2',\n# '../input/chuviettay/CHUHOA/5',\n# '../input/chuviettay/CHUHOA/7',\n# '../input/chuviettay/CHUHOA/10'\n# ]\n# print(subfolders)\n# print(len(subfolders))","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.338921Z","iopub.execute_input":"2023-01-13T10:36:27.339482Z","iopub.status.idle":"2023-01-13T10:36:27.343099Z","shell.execute_reply.started":"2023-01-13T10:36:27.339192Z","shell.execute_reply":"2023-01-13T10:36:27.342233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# subfolders= ['../input/chuviettay/CHUHOA/14',\n# '../input/chuviettay/CHUHOA/15',\n# '../input/chuviettay/CHUHOA/16',\n# '../input/chuviettay/CHUHOA/17',\n# '../input/chuviettay/CHUHOA/18',\n# '../input/chuviettay/CHUHOA/19',\n# '../input/chuviettay/CHUHOA/20'\n# ]\n# print(subfolders)\n# print(len(subfolders))","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.344919Z","iopub.execute_input":"2023-01-13T10:36:27.345458Z","iopub.status.idle":"2023-01-13T10:36:27.352301Z","shell.execute_reply.started":"2023-01-13T10:36:27.345408Z","shell.execute_reply":"2023-01-13T10:36:27.351696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# subfolders= ['../input/chuviettay/CHUHOA/21',\n# '../input/chuviettay/CHUHOA/22',\n# '../input/chuviettay/CHUHOA/23',\n# '../input/chuviettay/CHUHOA/24',\n# '../input/chuviettay/CHUHOA/25'\n# ]\n# print(subfolders)\n# print(len(subfolders))","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.382360Z","iopub.execute_input":"2023-01-13T10:36:27.382831Z","iopub.status.idle":"2023-01-13T10:36:27.386359Z","shell.execute_reply.started":"2023-01-13T10:36:27.382603Z","shell.execute_reply":"2023-01-13T10:36:27.385488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# subfolders= ['../input/chuviettay/CHUHOA/61',\n# '../input/chuviettay/CHUHOA/62',\n# '../input/chuviettay/CHUHOA/63',\n# '../input/chuviettay/CHUHOA/64',\n# '../input/chuviettay/CHUHOA/65',\n# '../input/chuviettay/CHUHOA/66',\n# '../input/chuviettay/CHUHOA/67',\n# '../input/chuviettay/CHUHOA/68',\n# '../input/chuviettay/CHUHOA/69',\n# '../input/chuviettay/CHUHOA/70',\n# ]\n# print(subfolders)\n# print(len(subfolders))","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.387995Z","iopub.execute_input":"2023-01-13T10:36:27.388506Z","iopub.status.idle":"2023-01-13T10:36:27.396821Z","shell.execute_reply.started":"2023-01-13T10:36:27.388448Z","shell.execute_reply":"2023-01-13T10:36:27.396252Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" subfolders= ['../input/chuviettay/CHUHOA/71',\n '../input/chuviettay/CHUHOA/72',\n '../input/chuviettay/CHUHOA/73',\n '../input/chuviettay/CHUHOA/74',\n '../input/chuviettay/CHUHOA/75',\n '../input/chuviettay/CHUHOA/76',\n '../input/chuviettay/CHUHOA/77',\n '../input/chuviettay/CHUHOA/78',\n '../input/chuviettay/CHUHOA/79',\n '../input/chuviettay/CHUHOA/80',\n ]\nprint(subfolders)\nprint(len(subfolders))","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.398472Z","iopub.execute_input":"2023-01-13T10:36:27.398774Z","iopub.status.idle":"2023-01-13T10:36:27.412719Z","shell.execute_reply.started":"2023-01-13T10:36:27.398713Z","shell.execute_reply":"2023-01-13T10:36:27.411782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Khởi tạo model theo model gốc**\n","metadata":{}},{"cell_type":"code","source":"kernel_start = time.perf_counter()\nkernel_time_limit = 60*60*8.5 #### running time\n\n# image size\nimg_size = 64\nchannels = 3\nimg_shape = (img_size, img_size, channels) # (64,64,3)\n\n# z(latent variable) size\nz_dim = 100\nz_shape = (z_dim,)\n\n# gradient penalty coefficient \"λ\"\npenaltyLambda = 10 # d_loss = f_loss - r_loss + λ･penalty\n\n# critic(discriminator) iterations per generator iteration\ntrainRatio = 5\n\nbatch_size = 32 # 16 or 64 better?\nrec_interval = 10000","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.423743Z","iopub.execute_input":"2023-01-13T10:36:27.423962Z","iopub.status.idle":"2023-01-13T10:36:27.430874Z","shell.execute_reply.started":"2023-01-13T10:36:27.423918Z","shell.execute_reply":"2023-01-13T10:36:27.430079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def build_generator():\n input = Input(shape=z_shape)\n x = Dense(8*img_size*img_size, activation=\"relu\")(input)\n x = Reshape((img_size//8, img_size//8, -1))(x)\n x = UpSampling2D((2, 2))(x)\n x = Conv2D(64*4, kernel_size=3, strides=1, padding=\"same\",\n use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5)(x, training=1)\n x = Conv2D(64*4, kernel_size=3, strides=1, padding=\"same\", use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5, )(x, training=1)\n x = Activation(\"relu\")(x)\n x = Conv2D(64*4, kernel_size=3, strides=1, padding=\"same\", use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5)(x, training=1)\n x = Activation(\"relu\")(x)\n x = UpSampling2D((2, 2))(x)\n x = Conv2D(64*2, kernel_size=3, strides=1, padding=\"same\",\n use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5)(x, training=1)\n x = Conv2D(64*2, kernel_size=3, strides=1, padding=\"same\", use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5, )(x, training=1)\n x = Activation(\"relu\")(x)\n x = Conv2D(64*2, kernel_size=3, strides=1, padding=\"same\", use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5)(x, training=1)\n x = Activation(\"relu\")(x)\n x = UpSampling2D((2, 2))(x)\n x = Conv2D(64*1, kernel_size=3, strides=1, padding=\"same\",\n use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5)(x,training=1)\n x = Conv2D(64*1, kernel_size=3, strides=1, padding=\"same\", use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5, )(x, training=1)\n x = Activation(\"relu\")(x)\n x = Conv2D(64*1, kernel_size=3, strides=1, padding=\"same\", use_bias=False)(x)\n x = BatchNormalization(momentum=0.9, epsilon=1e-5)(x, training=1)\n x = Activation(\"relu\")(x)\n x = Conv2D(3, kernel_size=3, strides=1, padding=\"same\", activation=\"tanh\",\n use_bias=False,)(x)\n\n model = Model(input, x)\n# print(\"●generator\")\n# model.summary()\n return model","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.549483Z","iopub.execute_input":"2023-01-13T10:36:27.549970Z","iopub.status.idle":"2023-01-13T10:36:27.567054Z","shell.execute_reply.started":"2023-01-13T10:36:27.549735Z","shell.execute_reply":"2023-01-13T10:36:27.566456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def build_discriminator():\n input = Input(shape=img_shape)\n x = Conv2D(64, kernel_size=4, strides=2, padding=\"same\", use_bias=False)(input)\n x = LeakyReLU(0.2)(x)\n x = Conv2D(128, kernel_size=4, strides=2, padding=\"same\", use_bias=False)(x)\n x = LeakyReLU(0.2)(x)\n x = Conv2D(256, kernel_size=4, strides=2, padding=\"same\", use_bias=False)(x)\n x = Conv2D(256, kernel_size=3, strides=1, padding=\"same\", use_bias=False)(x)\n x = LeakyReLU(0.2)(x)\n x = Conv2D(256, kernel_size=3, strides=1, padding=\"same\", use_bias=False)(x)\n x = LeakyReLU(0.2)(x)\n x = Conv2D(512, kernel_size=4, strides=2, padding=\"same\", use_bias=False)(x)\n x = LeakyReLU(0.2)(x)\n x = Conv2D(1, kernel_size=1, strides=1, padding=\"same\", use_bias=False)(x)\n x = Flatten()(x)\n x = Dense(units=1, activation=None)(x) # activation = None\n \n model = Model(input, x)\n# print(\"●discriminator\")\n# model.summary()\n return model","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.569325Z","iopub.execute_input":"2023-01-13T10:36:27.569800Z","iopub.status.idle":"2023-01-13T10:36:27.582257Z","shell.execute_reply.started":"2023-01-13T10:36:27.569696Z","shell.execute_reply":"2023-01-13T10:36:27.581653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def build_WGANgp(generator, discriminator):\n #### model\n # generator image(fake image)\n z = Input(shape=z_shape)\n f_img = generator(z)\n f_out = discriminator(f_img)\n # real image\n r_img = Input(shape=img_shape)\n r_out = discriminator(r_img)\n # average image\n epsilon = K.placeholder(shape=(None,1,1,1))\n a_img = Input(shape=(img_shape),\n tensor = epsilon * r_img + (1-epsilon) * f_img)\n a_out = discriminator(a_img)\n\n #### loss\n # original critic(discriminator) loss\n r_loss = K.mean(r_out)\n f_loss = K.mean(f_out)\n # gradient penalty \n grad_mixed = K.gradients(a_out, [a_img])[0]\n norm_grad_mixed = K.sqrt(K.sum(K.square(grad_mixed), axis=[1,2,3]))\n grad_penalty = K.mean(K.square(norm_grad_mixed -1))\n penalty = penaltyLambda * grad_penalty\n # d loss\n d_loss = f_loss - r_loss + penalty\n \n #### discriminator update function\n d_updates = Adam(lr=1e-4, beta_1=0.5, beta_2=0.9). \\\n get_updates(discriminator.trainable_weights,[],d_loss)\n d_train = K.function([r_img, z, epsilon],\n [r_loss, f_loss, penalty, d_loss],\n d_updates)\n \n #### generator update function\n g_loss = -1. * f_loss\n g_updates = Adam(lr=1e-4, beta_1=0.5, beta_2=0.9). \\\n get_updates(generator.trainable_weights,[],g_loss)\n g_train = K.function([z], [g_loss], g_updates)\n\n return g_train, d_train","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.583415Z","iopub.execute_input":"2023-01-13T10:36:27.584024Z","iopub.status.idle":"2023-01-13T10:36:27.597025Z","shell.execute_reply.started":"2023-01-13T10:36:27.583823Z","shell.execute_reply":"2023-01-13T10:36:27.596445Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" def sumple_images(imgs, rows=3, cols=3, figsize=(12,10)):\n fig, axes = plt.subplots(nrows=rows, ncols=cols, figsize=figsize)\n for indx, axis in enumerate(axes.flatten()):\n img = image.array_to_img(imgs[indx]) # ndarray → PIL\n imgplot = axis.imshow(img)\n axis.set_axis_off()\n plt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.598169Z","iopub.execute_input":"2023-01-13T10:36:27.598431Z","iopub.status.idle":"2023-01-13T10:36:27.612154Z","shell.execute_reply.started":"2023-01-13T10:36:27.598381Z","shell.execute_reply":"2023-01-13T10:36:27.611426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" def loadImage(fPath, resize = True):\n img = cv2.imread(fPath)\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) # BGR to RGB\n img = cv2.resize(img, (img_size, img_size)) # resize\n return img","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.614439Z","iopub.execute_input":"2023-01-13T10:36:27.614856Z","iopub.status.idle":"2023-01-13T10:36:27.622632Z","shell.execute_reply.started":"2023-01-13T10:36:27.614686Z","shell.execute_reply":"2023-01-13T10:36:27.621624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Dùng vòng lặp for để chạy từng folder trong danh sách subfolders\n# \n# Mỗi folder được train bởi 1500 epochs \"while iteration <= 1500\"\nChọn 1500 epochs vì em thấy hình sinh ra đã đẹp và có thể đủ thời gian chạy trên kaggle (kaggle giới hạn thời gian dùng GPU là 38h) \n\nVà em không thể chuyển sang Google Colab vì các thư viện, môi trường của notebook \"WGAN-gp (keras)\" nằm ở 2019 và không còn hỗ trợ( import trên colab thì bị lỗi, dùng lệnh \"pip install\" cài theo phiên bản cũ cũng bị lỗi, cài thư viện bản mới thì xuất hiện lỗi ở code) --> Notebook này em dùng chức năng \"Copy & Edit\" trên notebook gốc nên chạy đc\n\n# Sau khi train xong tạo 1000 ảnh \"n = 1000","metadata":{}},{"cell_type":"code","source":"for folder in subfolders:\n \n DIR = os.getcwd()\n DIRimg = folder\n DIRout = '/kaggle/working/' + folder.split(\"/\")[-1]\n \n all_fNames = os.listdir(DIRimg)\n\n # image sample\n sample_ids = random.sample(range(len(all_fNames)), 9)\n fig, axes = plt.subplots(nrows=3, ncols=3, figsize=(12,10))\n for i, axis in enumerate(axes.flatten()):\n img = loadImage(os.path.join(DIRimg,\n all_fNames[sample_ids[i]]), resize=False)\n imgplot = axis.imshow(img)\n axis.set_title(all_fNames[sample_ids[i]])\n axis.set_axis_off()\n plt.tight_layout()\n \n \n # train data\n x_train = np.zeros((len(all_fNames),img_size,img_size,3))\n for i in tqdm(range(len(all_fNames))):\n path = os.path.join(DIRimg, all_fNames[i])\n x_train[i] = loadImage(path)\n\n x_train = x_train / 255.\n print(x_train.shape)\n \n \n \n # generator Model\n generator = build_generator()\n # discriminator Model\n discriminator = build_discriminator()\n # WGAN-gp Training Model\n G_train, D_train = build_WGANgp(generator, discriminator)\n \n \n \n # fixed z for confirmation of generated image\n z_fix = np.random.normal(0, 1, (64, z_dim)) \n\n # list for store learning progress data\n g_loss_list = []\n r_loss_list = []\n f_loss_list = []\n f_r_loss_list = []\n penalty_list = []\n d_loss_list = []\n\n # (0～1) → (-1～+1)\n X_train = (x_train.astype(np.float32) - 0.5) / 0.5\n \n \n \n \n iteration = 0\n while iteration <= 1500:\n \n #### Discriminator\n for j in range(trainRatio):\n # Generator in\n z = np.random.normal(0, 1, (batch_size, z_dim))\n # Generator out Images\n f_imgs = generator.predict(z)\n # Real Images\n idx = np.random.randint(0, X_train.shape[0], batch_size)\n r_imgs = X_train[idx]\n # train the discriminator\n epsilon = np.random.uniform(size = (batch_size, 1,1,1))\n r_loss, f_loss, penalty, d_loss = D_train([r_imgs, z, epsilon])\n\n #### Generator\n # Generator in\n z = np.random.normal(0, 1, (batch_size, z_dim))\n # train the generator\n g_loss = G_train([z])\n\n #### Record of learning progress\n # loss\n r_loss_list.append(r_loss)\n f_loss_list.append(f_loss)\n f_r_loss_list.append(f_loss - r_loss)\n penalty_list.append(penalty)\n d_loss_list.append(d_loss)\n # generated image sumple\n if (iteration in [100, 500,1000]) or (iteration % rec_interval == 0) or (iteration== 1500):\n print(f'iteration:{iteration} / d_loss:{d_loss:.3f} / g_loss:{sum(g_loss)/len(g_loss):.3f}')\n g_imgs = generator.predict(z_fix)\n imgs = g_imgs * 127.5 + 127.5\n sumple_images(imgs, rows=1, cols=7)\n plt.show()\n\n iteration += 1\n\n print(\"last iteration:\",iteration - 1)\n \n if os.path.exists(DIRout):\n shutil.rmtree(DIRout)\n if not os.path.exists(DIRout):\n os.mkdir(DIRout)\n # generate images for submit\n n = 1000\n batch = 64\n for i in tqdm(range(0, n, batch)):\n z = np.random.normal(0,1,size=(batch, z_dim))\n g_imgs = generator.predict(z)\n imgs = g_imgs * 127.5 + 127.5\n for j in range(batch):\n img = image.array_to_img(imgs[j]) # ndarray → PIL \n img.save(os.path.join(DIRout, 'image_' + str(i+j+1).zfill(5) + '.png'))\n if i+j+1 == n:\n break\n print(len(os.listdir(DIRout)))\n \n sumple_images(g_imgs, rows=5, cols=7, figsize=(12,8))\n \n filename = 'images_{}.zip'.format(folder)\n if os.path.exists(filename):\n os.remove(filename)\n shutil.make_archive(folder.split(\"/\")[-1], 'zip', DIRout)\n print(\"=====================================================\")","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:36:27.624285Z","iopub.execute_input":"2023-01-13T10:36:27.624769Z","iopub.status.idle":"2023-01-13T12:13:40.163901Z","shell.execute_reply.started":"2023-01-13T10:36:27.624721Z","shell.execute_reply":"2023-01-13T12:13:40.163092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"shutil.make_archive('dataset31_80', 'zip','/kaggle/working/')","metadata":{"execution":{"iopub.status.busy":"2023-01-13T12:28:05.114107Z","iopub.execute_input":"2023-01-13T12:28:05.114465Z","iopub.status.idle":"2023-01-13T12:31:18.451310Z","shell.execute_reply.started":"2023-01-13T12:28:05.114408Z","shell.execute_reply":"2023-01-13T12:31:18.449562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!tar -czvf /kaggle/working/my_compressed_folder.tar.gz /kaggle/working/\n","metadata":{"execution":{"iopub.status.busy":"2023-01-13T13:22:19.484154Z","iopub.execute_input":"2023-01-13T13:22:19.486237Z","iopub.status.idle":"2023-01-13T13:22:20.439940Z","shell.execute_reply.started":"2023-01-13T13:22:19.486184Z","shell.execute_reply":"2023-01-13T13:22:20.439133Z"},"trusted":true},"execution_count":null,"outputs":[]}]}