{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://storage.googleapis.com/kaggle-competitions/kaggle/36073/logos/header.png?t=2022-04-12-20-53-30)","metadata":{}},{"cell_type":"markdown","source":"# 1. Definition of Augraphy \n\n[Augraphy](https://github.com/sparkfish/augraphy) is an open source Python package that creates multiple copies of original documents though an augmentation pipeline that randomly distorts each copy -- degrading the clean version into dirty and realistic copies rendered through synthetic paper printing, faxing, scanning and copy machine processes.\n\nAugraphy is focused on supporting problems related to automation of document images such as OCR, form recognition, form data extraction, document classification, barcode decoding, denoising, document restoration, identity document data extraction, document cropping, etc. Eventually, Augraphy will be able to support photo OCR problems with augmentations designed to emulate camera phone distortions.","metadata":{}},{"cell_type":"markdown","source":"# 2. Application of Augraphy\n## Image Augmentation ","metadata":{}},{"cell_type":"code","source":"!pip install augraphy","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:45:27.767032Z","iopub.execute_input":"2022-06-10T02:45:27.767515Z","iopub.status.idle":"2022-06-10T02:45:40.336362Z","shell.execute_reply.started":"2022-06-10T02:45:27.767445Z","shell.execute_reply":"2022-06-10T02:45:40.335378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport sys\nimport cv2\nimport glob\nimport random\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom augraphy import *","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:45:49.694271Z","iopub.execute_input":"2022-06-10T02:45:49.694968Z","iopub.status.idle":"2022-06-10T02:45:50.652194Z","shell.execute_reply.started":"2022-06-10T02:45:49.694927Z","shell.execute_reply":"2022-06-10T02:45:50.651181Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BASE_DIR = \"/kaggle/input/denoising-shabby-pages\"\nimage_paths = glob.glob(BASE_DIR+\"/train_cleaned/train_cleaned/*.png\")\ntry:\n    index_list = random.sample(range(0, len(image_paths)), 3)\nexcept:\n    print('Sample size exceeded population size.')\n    sys.exit(1)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:45:54.391163Z","iopub.execute_input":"2022-06-10T02:45:54.391968Z","iopub.status.idle":"2022-06-10T02:45:54.45983Z","shell.execute_reply.started":"2022-06-10T02:45:54.39193Z","shell.execute_reply":"2022-06-10T02:45:54.459058Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_pipeline():\n\n    ################################################################################\n    # CONTROLLING VARIATION\n    #\n    # We can control the outputs of the pipeline by setting values here.\n    ################################################################################\n    \n    # Dithering.dither_type can be 'ordered' for ordered dithering or another string for floyd-steinberg dithering\n    dithering_dither_type = random.choice([\"ordered\", \"floyd-steinberg\"])\n    # Dithering.order determines the dimensions of the threshold map\n    dithering_order = random.choice(range(3,10))\n    # dithering.p is the probability to run this augmentation\n    dithering_p = (random.random()>0.7) * 1\n    \n    # InkBleed.intensity_range is a tuple with bounds for bleed intensity to be selected from\n    inkbleed_intensity_range = (0.1, 0.4)\n    # InkBleed.color_range is a tuple with bounds for color noise\n    inkbleed_color_range = (0, 32)\n    # InkBleed.kernel_size determines the radius of the bleed effect\n    inkbleed_kernel_size = random.choice([(7,7), (5, 5), (3,3)])\n    # InkBleed.severity determines significance of bleed effect\n    inkbleed_severity=(0.4, 0.6)\n    \n    # BleedThrough.intensity_range is a tuple with bounds for bleed intensity to be selected from\n    bleedthrough_intensity_range=(0.1, 0.2)\n    # BleedThrough.color_range is a tuple with bounds for color noise\n    bleedthrough_color_range=(32, 224)\n    # BleedThrough.ksize is a tuple of height/width pairs for sampling kernel size\n    bleedthrough_ksize=(17, 17)\n    # BleedThrough.sigmaX is the standard deviation of the kernel along the x-axis\n    bleedthrough_sigmaX=0\n    # BleedThrough.alpha is the intensity of the bleeding effect, recommended 0.1-0.5\n    bleedthrough_alpha=random.uniform(0.05,0.1)\n    # BleedThrough.offsets is a pair of x and y distances to shift the bleed with\n    bleedthrough_offsets=(10, 20)\n    \n    # Letterpress.n_samples is a tuple determining how many points to generate per cluster\n    letterpress_n_samples=(100, 300)\n    # Letterpress.n_clusters is a tuple determining how many clusters to generate\n    letterpress_n_clusters=(300, 400)\n    # Letterpress.std_range is a pair of ints determining the std deviation range in each blob\n    letterpress_std_range=(500, 3000)\n    # Letterpress.value_range determines values that points in the blob are sampled from\n    letterpress_value_range=(150, 224)\n    # Letterpress.value_threshold_range is the minimum pixel value to enable the effect (e.g. 128)\n    letterpress_value_threshold_range=(96, 128)\n    # Letterpress.blur enables blur in the noise mask\n    letterpress_blur=1\n    # Letterpress.p is the probability to run this augmentation\n    letterpress_p = (random.random()>0.5) * 1\n    \n    # LowInkLines.count_range is a pair determining how many lines should be drawn\n    lowinkrandomlines_count_range = (3, 12)\n    lowinkperiodiclines_count_range = (1, 2)\n    # LowInkLines.use_consistent_lines is false if we should vary the width and alpha of lines\n    lowinkrandomlines_use_consistent_lines=random.choice([True, False])\n    lowinkperiodiclines_use_consistent_lines=random.choice([True, False])\n    # LowInkPeriodicLines.period_range is a pair determining how wide the gap between lines can be\n    lowinkperiodiclines_period_range=(16, 32)\n    \n    # PaperFactory.texture_path is the directory to pull textures from\n    paperfactory_texture_path = \"./paper_textures\"\n    \n    # NoiseTexturize.sigma_range defines bounds of noise fluctuations\n    noisetexturize_sigma_range = (3, 10)\n    # NoiseTexturize.turbulence_range defines how quickly big patterns are replaced with small ones; lower means more iterations\n    noisetexturize_turbulence_range = (2, 5)\n    \n    # BrightnessTexturize.range determines the value range of samples for the brightness matrix\n    brightnesstexturize_range = (0.9, 0.99)\n    # BrightnessTexturize.deviation is additional variation for the uniform sample\n    brightnesstexturize_deviation = 0.03\n    \n    # Brightness.range is a pair of floats determining the brightness delta\n    brightness_range = (0.9, 1.1)\n    \n    # PageBorder.side determines the page edge of the effect\n    pageborder_side = random.choice([\"left\", \"top\", \"bottom\", \"right\"])\n    # PageBorder.width_range determines border thickness\n    pageborder_width_range = (5, 30)\n    # PageBorder.pages determines how many page shadows to render\n    pageborder_pages = None # internally this is random.randint(2, 7)\n    # PageBorder.intensity_range determines the noises of the page shadow\n    pageborder_intensity_range = (0.4, 0.8)\n    # PageBorder.curve_frequency determines frequency of curvy page shadow\n    pageborder_curve_frequency = (2, 8)\n    # PageBorder.curve_height determines height of curvy page shadow\n    pageborder_curve_height = (2, 4)\n    # PageBorder.curve_length_one_side determines Length for one side of generated curvy page shadow\n    pageborder_curve_length_one_side = (50, 100)\n    # PageBorder.value determines value of generated page shadow\n    pageborder_value = (32, 150)\n    \n    # DirtyRollers.line_width_range determines the width of roller lines\n    dirtyrollers_line_width_range = (2, 32)\n    # DirtyRollers.scanline_type changes the background of lines\n    dirtyrollers_scanline_type = 0\n    # DirtyRollers.p is the probability to run this augmentation\n    dirtyrollers_p = (random.random()>0.5) * 1\n\n    # LightingGradient.mask_size determines how big the mask should be\n    lightinggradient_light_position = None\n    # LightingGradient.direction indicates the rotation degree of the light strip\n    lightinggradient_direction = None\n    # LightingGradient.max_brightness and LightingGradient.min_brightness set bounds for how much brightness change will happen\n    lightinggradient_max_brightness = 196\n    lightinggradient_min_brightness = 0\n    # LightingGradient.mode is linear or gaussian depending on how light should decay\n    lightinggradient_mode = random.choice([\"linear_dynamic\", \"linear_static\", \"gaussian\"])\n    # LightingGradient.linear_decay_rate is only valid in linear static mode\n    lightinggradient_linear_decay_rate = None\n    # LightingGradient.transparency gives the transparency of the input image\n    lightinggradient_transparency = None\n    \n    # DirtyDrum.line_width_range determines the range from which drum line widths are sampled\n    dirtydrum_line_width_range = (1, 6)\n    # concentration of dirty drum line\n    dirtydrum_line_concentration = np.random.uniform(0.05, 0.15)\n    # DirtyDrum.direction is 0 for horizontal, 1 for vertical, 2 for both\n    dirtydrum_direction = random.randint(0,2)\n    # DirtyDrum.noise_intensity changes how significant the effect is, recommended 0.8-1.0\n    dirtydrum_noise_intensity = np.random.uniform(0.6, 0.95)\n    # DirtyDrum.noise_value determine the intensity of dirty drum noise\n    dirtydrum_noise_value = (128, 224)\n    # DirtyDrum.ksize is a tuple of height/width pairs from which to sample kernel size\n    dirtydrum_ksize = random.choice([(3, 3),(5,5),(7,7)])\n    # DirtyDrum.sigmaX is the stdev of the kernel in the x direction\n    dirtydrum_sigmaX = 0\n    # DirtyDrum.p is the probability to run this augmentation\n    dirtydrum_p = (random.random()>0.5) * 1\n\n    # SubleNoise.range gives the variation range for sampling noise\n    subtlenoise_range = 10\n    \n    # Jpeg.quality_range determines the range from which to sample compression level\n    jpeg_quality_range = (25, 95)\n    \n    # Markup.num_lines_range determines how many lines get marked up\n    markup_num_lines_range=(2, 7)\n    # Markup.length_range determines the relative length of the drawn effect\n    markup_length_range=(0.5, 1)\n    # Markup.thickness_range determines the thickness of the drawn effect\n    markup_thickness_range=(1, 2)\n    # Markup.type determines the style of effect\n    markup_type=random.choice([\"strikethrough\", \"crossed\", \"highlight\", \"underline\"])\n    # Markup.color is the color of the ink used to markup\n    markup_color=(random.randint(0,256),\n                  random.randint(0,256),\n                  random.randint(0,256))\n    # Markup.single_word_mode determines whether to draw across multiple words\n    markup_single_word_mode=random.choice([True, False])\n    # Markup.repetitions determines the number of times the effect is drawn\n    markup_repetitions=random.randint(1,2) if markup_type == \"highlight\" else 1\n    # Markup.p is the probability to run this augmentation\n    markup_p = (random.random()>0.5) * 1\n    \n    # PencilScribbles.size_range determines the size of scribbles to draw\n    pencilscribbles_size_range=(100, 700)\n    # PencilScribbles.count_range determines how many scribbles to draw\n    pencilscribbles_count_range=(1, 6)\n    # PencilScribbles.stroke_count_range determines how many strokes per scribble\n    pencilscribbles_stroke_count_range=(1, 1)\n    # PencilScribbles.thickness_range determines how thick strokes are\n    pencilscribbles_thickness_range=(2, 6)\n    # PencilScribbles.brightness_change is the brightness value of each stroke\n    pencilscribbles_brightness_change=random.randint(64,224)\n    # PencilScribbles.p is the probability to run this augmentation\n    pencilscribbles_p = (random.random()>0.5) * 1\n\n    # BindingsAndFasteners.overlay_types can be min, max, or mix\n    bindingsandfasteners_overlay_types = \"darken\"\n    # BindingsAndFasteners.foreground is the path to fg image or the image itself\n    bindingsandfasteners_foreground = None\n    # BindingsAndFasteners.effect_type is \"punch_holes\", \"binding_holes\", or \"clips\"\n    bindingsandfasteners_effect_type = random.choice([\"punch_holes\", \"binding_holes\", \"clips\"])\n    # BindingsAndFasteners.ntimes gives how many fg images to draw\n    bindingsandfasteners_ntimes = (10,20) if bindingsandfasteners_effect_type == \"binding_holes\" else (2,3)\n    # BindingsAndFasteners.nscales is the scale of the fg image size\n    bindingsandfasteners_nscales = (0.9,1)\n    # BindingsAndFasteners.edge gives the edge to place the images on\n    bindingsandfasteners_edge = \"random\"\n    # BindingsAndFasteners.edge_offset is how far from the page edge to draw\n    bindingsandfasteners_edge_offset = (10,50)\n    \n    # BadPhotoCopy.mask is a mask of noise to generate the effect with\n    badphotocopy_mask=None\n    # BadPhotoCopy.noise_type determines which mask pattern to use\n    badphotocopy_noise_type=-1  # -1 is random\n    # BadPhotoCopy.noise_side determines where to add noise\n    badphotocopy_noise_side=\"random\"\n    # BadPhotoCopy.noise_iteration determines how many times to apply noise in the mask\n    badphotocopy_noise_iteration=(1, 2)\n    # BadPhotoCopy.noise_size determines the scale of noise in the mask\n    badphotocopy_noise_size=(1, 2)\n    # BadPhotoCopy.noise_value determines the intensity of the noise\n    badphotocopy_noise_value=(128, 196)\n    # BadPhotoCopy.noise_sparsity determines the sparseness of noise\n    badphotocopy_noise_sparsity=(0.3, 0.6)\n    # BadPhotoCopy.noise_concentration determines the concentration of noise\n    badphotocopy_noise_concentration=(0.1, 0.5)\n    # BadPhotoCopy.blur_noise determines whether or or not to add blur\n    badphotocopy_blur_noise=random.choice([True,False])\n    # BadPhotoCopy.blur_noise_kernel gives the dimensions for the noise kernel\n    badphotocopy_blur_noise_kernel=random.choice([(3, 3), (5, 5), (7, 7)])\n    # BadPhotoCopy.wave_pattern enables the wave pattern in the noise mask\n    badphotocopy_wave_pattern=random.choice([True,False])\n    # BadPhotoCopy.edge_effect adds the Sobel edge effect to the noise mask\n    badphotocopy_edge_effect=random.choice([True,False])\n    # BadPhotoCopy.p is the probability to run this augmentation\n    badphotocopy_p = (random.random()>0.5) * 1\n    \n    # Gamma.range is an interval from which to sample a gamma shift\n    gamma_range = (0.8, 1.2)\n    \n    # Geometric.scale is a pair determining how to scale the image\n    geometric_scale = (1, 1)\n    # Geometric.translation is a pair determining where to translate the image\n    geometric_translation = (0, 0)\n    # Geometric.fliplr flips the image left and right\n    geometric_fliplr = 0\n    # Geometric.flipud flips the image up and down\n    geometric_flipud = 0\n    # Geometric.crop is a tuple of four points giving the corners of a crop region\n    geometric_crop = ()\n    # Geometric.rotate_range is a pair determining the rotation angle sample range\n    geometric_rotate_range = (0, 0)\n    \n    # Faxify.scale_range is a pair of ints determining the scaling magnitude\n    faxify_scale_range=(0.3, 0.6)\n    # Faxify.monochrome determines whether the image will get the binarization effect\n    faxify_monochrome=random.choice([0, 1])\n    # Faxify.monochrome_method is the binarization method for applying the effect\n    faxify_monochrome_method=\"random\"\n    # Faxify.monochrome_argument is the input arguments to the monochrome method\n    faxify_monochrome_arguments = {}\n    # Faxify.halftone determins whether the image will get halftone effect\n    faxify_halftone = random.choice((0, 1))\n    # Faxify.invert determines whether to invert the grayscale value in halftone\n    faxify_invert=1\n    # Faxify.half_kernel_size is the half Gaussian kernel size for halftone\n    faxify_half_kernel_size=(1, 1)\n    # Faxify.angle is the angle of the halftone effect\n    faxify_angle=(0, 360)\n    # Faxify.sigma is the sigma value of the Gaussian kernel in the halftone effect\n    faxify_sigma=(1,3)\n    \n    # to prevent text becomes unreadable after the agumentation\n    if badphotocopy_p>0 or dirtyrollers_p>0 or dirtydrum_p>0 or markup_p>0 or pencilscribbles_p>0:\n        faxify_monochrome_method = \"grayscale\"\n    if dithering_p or faxify_halftone:\n        letterpress_p = 0\n        if dithering_p:\n            faxify_halftone = 0\n    \n    ################################################################################\n    # PIPELINE\n    #\n    # The default Augraphy pipeline is defined in parametric form here.\n    ################################################################################\n    \n    ink_phase = [\n        Dithering(dithering_dither_type,\n                  dithering_order,\n                  p=dithering_p),\n    \n        InkBleed(inkbleed_intensity_range,\n                 inkbleed_color_range,\n                 inkbleed_kernel_size,\n                 inkbleed_severity,\n                 p=0.5),\n    \n        BleedThrough(bleedthrough_intensity_range,\n                     bleedthrough_color_range,\n                     bleedthrough_ksize,\n                     bleedthrough_sigmaX,\n                     bleedthrough_alpha,\n                     bleedthrough_offsets,\n                     p=0.5),\n    \n        Letterpress(letterpress_n_samples,\n                    letterpress_n_clusters,\n                    letterpress_std_range,\n                    letterpress_value_range,\n                    letterpress_value_threshold_range,\n                    letterpress_blur,\n                    p=letterpress_p),\n    \n        OneOf(\n            [\n                LowInkRandomLines(lowinkrandomlines_count_range,\n                                  lowinkrandomlines_use_consistent_lines),\n    \n                LowInkPeriodicLines(lowinkperiodiclines_count_range,\n                                    lowinkperiodiclines_period_range,\n                                    lowinkperiodiclines_use_consistent_lines),\n            ],\n        ),\n    ]\n    \n    paper_phase = [\n        PaperFactory(paperfactory_texture_path,\n                     p=0.5),\n        OneOf(\n            [\n                AugmentationSequence(\n                    [\n                        NoiseTexturize(noisetexturize_sigma_range,\n                                       noisetexturize_turbulence_range),\n    \n                        BrightnessTexturize(brightnesstexturize_range,\n                                            brightnesstexturize_deviation),\n                    ],\n                ),\n                AugmentationSequence(\n                    [\n                        BrightnessTexturize(brightnesstexturize_range,\n                                            brightnesstexturize_deviation),\n                        NoiseTexturize(noisetexturize_sigma_range,\n                                       noisetexturize_turbulence_range),\n                    ],\n                ),\n            ],\n            p=0.5,\n        ),\n    \n        Brightness(brightness_range,\n                   p=0.5),\n    ]\n    \n    post_phase = [\n        BrightnessTexturize(brightnesstexturize_range,\n                            brightnesstexturize_deviation,\n                            p=0.5),\n    \n        OneOf(\n            [\n                PageBorder(pageborder_side,\n                           pageborder_width_range,\n                           pageborder_pages,\n                           pageborder_intensity_range,\n                           pageborder_curve_frequency,\n                           pageborder_curve_height,\n                           pageborder_curve_length_one_side,\n                           pageborder_value),\n\n                DirtyRollers(dirtyrollers_line_width_range,\n                             dirtyrollers_scanline_type,\n                             p=dirtyrollers_p)\n            ],\n            p=0.5),\n    \n        OneOf(\n            [\n                LightingGradient(lightinggradient_light_position,\n                                 lightinggradient_direction,\n                                 lightinggradient_max_brightness,\n                                 lightinggradient_min_brightness,\n                                 lightinggradient_mode,\n                                 lightinggradient_linear_decay_rate,\n                                 lightinggradient_transparency),\n                                 \n                Brightness(brightness_range)\n            ],\n            p=0.5),\n    \n        DirtyDrum(dirtydrum_line_width_range,\n                  dirtydrum_line_concentration,\n                  dirtydrum_direction,\n                  dirtydrum_noise_intensity,\n                  dirtydrum_noise_value,\n                  dirtydrum_ksize,\n                  dirtydrum_sigmaX,\n                  p=dirtydrum_p),\n    \n        SubtleNoise(subtlenoise_range,\n                    p=0.5),\n    \n        Jpeg(jpeg_quality_range,\n             p=0.5),\n    \n        Markup(markup_num_lines_range,\n               markup_length_range,\n               markup_thickness_range,\n               markup_type,\n               markup_color,\n               markup_single_word_mode,\n               markup_repetitions,\n               p=markup_p),\n    \n        PencilScribbles(pencilscribbles_size_range,\n                        pencilscribbles_count_range,\n                        pencilscribbles_stroke_count_range,\n                        pencilscribbles_thickness_range,\n                        pencilscribbles_brightness_change,\n                        p=pencilscribbles_p),\n    \n        BindingsAndFasteners(bindingsandfasteners_overlay_types,\n                             bindingsandfasteners_foreground,\n                             bindingsandfasteners_effect_type,\n                             bindingsandfasteners_ntimes,\n                             bindingsandfasteners_nscales,\n                             bindingsandfasteners_edge,\n                             bindingsandfasteners_edge_offset,\n                             p=0.5),\n    \n        BadPhotoCopy(badphotocopy_mask,\n                     badphotocopy_noise_type,\n                     badphotocopy_noise_side,\n                     badphotocopy_noise_iteration,\n                     badphotocopy_noise_size,\n                     badphotocopy_noise_value,\n                     badphotocopy_noise_sparsity,\n                     badphotocopy_noise_concentration,\n                     badphotocopy_blur_noise,\n                     badphotocopy_blur_noise_kernel,\n                     badphotocopy_wave_pattern,\n                     badphotocopy_edge_effect,\n                     p=badphotocopy_p),\n    \n        Gamma(gamma_range,\n              p=0.5),\n    \n        Geometric(geometric_scale,\n                  geometric_translation,\n                  geometric_fliplr,\n                  geometric_flipud,\n                  geometric_crop,\n                  geometric_rotate_range,\n                  p=0.5),\n    \n        Faxify(faxify_scale_range,\n               faxify_monochrome,\n               faxify_monochrome_method,\n               faxify_monochrome_arguments,\n               faxify_halftone,\n               faxify_invert,\n               faxify_half_kernel_size,\n               faxify_angle,\n               faxify_sigma,\n               p=0.5),\n    ]\n    \n    \"\"\" This makes things easier.\"\"\"\n    return AugraphyPipeline(ink_phase,paper_phase,post_phase)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:45:58.950627Z","iopub.execute_input":"2022-06-10T02:45:58.951292Z","iopub.status.idle":"2022-06-10T02:45:59.000051Z","shell.execute_reply.started":"2022-06-10T02:45:58.951257Z","shell.execute_reply":"2022-06-10T02:45:58.999351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pipeline = get_pipeline()\n\nfig, ax = plt.subplots(nrows=1, ncols=6, figsize=(30, 16))\nfor i, index in enumerate(index_list):\n    im = cv2.imread(image_paths[index])\n    data = pipeline.augment(im)[\"output\"]\n    \n    ax[2*i].set_title(\"Original Image\", size=20)\n    ax[2*i].imshow(im)\n    ax[2*i+1].set_title(\"Augmented Image\", size=20)\n    ax[2*i+1].imshow(data)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:47:40.011628Z","iopub.execute_input":"2022-06-10T02:47:40.011989Z","iopub.status.idle":"2022-06-10T02:47:54.474975Z","shell.execute_reply.started":"2022-06-10T02:47:40.011959Z","shell.execute_reply":"2022-06-10T02:47:54.474301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Denoising Shabby Pages in OCR\n\nThe [Denoising ShabbyPages](https://www.kaggle.com/competitions/denoising-shabby-pages/overview) competition is one to denoise the shabby pages imposed on them by Augraphy.\n\nThe following model can be used to reverse the effects by Augraphy.\n\nhttps://www.kaggle.com/code/deepdreamx/denoising-by-pix2pix","metadata":{}},{"cell_type":"code","source":"from glob import glob\nimport numpy as np\nfrom matplotlib import pylab as plt\nimport cv2\nimport tensorflow as tf\nprint(tf.__version__)\nimport tensorflow.keras.layers as layers\nfrom tensorflow.keras.models import Model\nimport time\nimport os\nimport os\nimport cv2\nimport glob\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:51:11.022853Z","iopub.execute_input":"2022-06-10T02:51:11.023765Z","iopub.status.idle":"2022-06-10T02:51:16.303239Z","shell.execute_reply.started":"2022-06-10T02:51:11.023724Z","shell.execute_reply":"2022-06-10T02:51:16.302366Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BASE_DIR = \"/kaggle/input/denoising-shabby-pages\"\ncleaned_img_paths = sorted(glob.glob(BASE_DIR+\"/train_cleaned/train_cleaned/*.png\"))\ntrain_shabby_paths = sorted(glob.glob(BASE_DIR+\"/train/train/*.png\"))\nlen(cleaned_img_paths),len(train_shabby_paths)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:51:45.220442Z","iopub.execute_input":"2022-06-10T02:51:45.220824Z","iopub.status.idle":"2022-06-10T02:51:45.235844Z","shell.execute_reply.started":"2022-06-10T02:51:45.220789Z","shell.execute_reply":"2022-06-10T02:51:45.235189Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tensorflow as tf\n\nimport os\nimport pathlib\nimport time\nimport datetime\n\nfrom matplotlib import pyplot as plt\nfrom IPython import display","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:51:47.928065Z","iopub.execute_input":"2022-06-10T02:51:47.92843Z","iopub.status.idle":"2022-06-10T02:51:47.933835Z","shell.execute_reply.started":"2022-06-10T02:51:47.928398Z","shell.execute_reply":"2022-06-10T02:51:47.931966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_image = tf.io.read_file(str(train_shabby_paths[2]))\nsample_image = tf.io.decode_jpeg(sample_image)\nprint(sample_image.shape)\nplt.figure()\nplt.imshow(sample_image)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:51:52.22993Z","iopub.execute_input":"2022-06-10T02:51:52.230279Z","iopub.status.idle":"2022-06-10T02:51:54.974205Z","shell.execute_reply.started":"2022-06-10T02:51:52.230249Z","shell.execute_reply":"2022-06-10T02:51:54.973388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load(input_image, real_image):\n    # Read and decode an image file to a uint8 tensor\n    input_image = tf.io.read_file(input_image)\n    #input_image = tf.io.decode_jpeg(input_image)\n    input_image = tf.io.decode_image(input_image, expand_animations = False, dtype=tf.float32, channels=3)*255.0\n\n    real_image = tf.io.read_file(real_image)\n    #real_image = tf.io.decode_jpeg(real_image)\n    real_image = tf.io.decode_image(real_image, expand_animations = False, dtype=tf.float32, channels=1)*255.0\n\n#     real_image = tf.io.decode_image(real_image, expand_animations = False, dtype=tf.float32, channels=3)*255.0\n\n    # Convert both images to float32 tensors\n    input_image = tf.cast(input_image, tf.float32)\n    real_image = tf.cast(real_image, tf.float32)\n    print(input_image.shape, real_image.shape)\n\n    return input_image, real_image","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:52:00.904239Z","iopub.execute_input":"2022-06-10T02:52:00.905381Z","iopub.status.idle":"2022-06-10T02:52:00.912107Z","shell.execute_reply.started":"2022-06-10T02:52:00.905303Z","shell.execute_reply":"2022-06-10T02:52:00.911286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"inp, re = load(train_shabby_paths[1],cleaned_img_paths[1])\n# Casting to int for matplotlib to display the images\n#inp\nplt.figure()\nplt.imshow(inp / 255.0)\nplt.figure()\nplt.imshow(re / 255.0)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:52:08.917131Z","iopub.execute_input":"2022-06-10T02:52:08.917832Z","iopub.status.idle":"2022-06-10T02:52:09.367872Z","shell.execute_reply.started":"2022-06-10T02:52:08.917789Z","shell.execute_reply":"2022-06-10T02:52:09.367113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# The facade training set consist of 400 images\nBUFFER_SIZE = 400\n# The batch size of 1 produced better results for the U-Net in the original pix2pix experiment\nBATCH_SIZE = 1\n# Each image is 256x256 in size\nIMG_WIDTH = 256\nIMG_HEIGHT = 256\n\ndef resize(input_image, real_image, height, width):\n  input_image = tf.image.resize(input_image, [height, width],\n                                method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)\n  real_image = tf.image.resize(real_image, [height, width],\n                               method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)\n\n  return input_image, real_image","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:52:59.703682Z","iopub.execute_input":"2022-06-10T02:52:59.704069Z","iopub.status.idle":"2022-06-10T02:52:59.710833Z","shell.execute_reply.started":"2022-06-10T02:52:59.704027Z","shell.execute_reply":"2022-06-10T02:52:59.709812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normalizing the images to [-1, 1]\ndef normalize(input_image, real_image):\n    input_image = (input_image / 127.5) - 1\n    real_image = (real_image / 127.5) - 1\n\n    return input_image, real_image","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:05.649625Z","iopub.execute_input":"2022-06-10T02:53:05.649988Z","iopub.status.idle":"2022-06-10T02:53:05.654553Z","shell.execute_reply.started":"2022-06-10T02:53:05.649957Z","shell.execute_reply":"2022-06-10T02:53:05.65355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_image_train(image_file):\n    input_image, real_image = load(image_file,tf.strings.regex_replace(image_file, \"train\", \"train_cleaned\"))\n    input_image, real_image = resize(input_image, real_image, 256, 256)\n    #input_image, real_image = random_jitter(input_image, real_image)\n    input_image, real_image = normalize(input_image, real_image)\n\n    return input_image, real_image","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:08.425824Z","iopub.execute_input":"2022-06-10T02:53:08.426434Z","iopub.status.idle":"2022-06-10T02:53:08.431788Z","shell.execute_reply.started":"2022-06-10T02:53:08.426399Z","shell.execute_reply":"2022-06-10T02:53:08.431029Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_image_test(image_file):\n    input_image, real_image = load(image_file)\n    input_image, real_image = resize(input_image, real_image,\n                                   IMG_HEIGHT, IMG_WIDTH)\n    input_image, real_image = normalize(input_image, real_image)\n\n    return input_image, real_image","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:12.307019Z","iopub.execute_input":"2022-06-10T02:53:12.307868Z","iopub.status.idle":"2022-06-10T02:53:12.314375Z","shell.execute_reply.started":"2022-06-10T02:53:12.30783Z","shell.execute_reply":"2022-06-10T02:53:12.311544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_dataset = tf.data.Dataset.list_files(train_shabby_paths)\ntrain_dataset = train_dataset.map(load_image_train,\n                                  num_parallel_calls=tf.data.AUTOTUNE)\ntrain_dataset = train_dataset.shuffle(BUFFER_SIZE)\ntrain_dataset = train_dataset.batch(BATCH_SIZE)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:15.614842Z","iopub.execute_input":"2022-06-10T02:53:15.615207Z","iopub.status.idle":"2022-06-10T02:53:16.092524Z","shell.execute_reply.started":"2022-06-10T02:53:15.615174Z","shell.execute_reply":"2022-06-10T02:53:16.091699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"OUTPUT_CHANNELS = 1","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:18.74799Z","iopub.execute_input":"2022-06-10T02:53:18.748681Z","iopub.status.idle":"2022-06-10T02:53:18.752761Z","shell.execute_reply.started":"2022-06-10T02:53:18.748625Z","shell.execute_reply":"2022-06-10T02:53:18.751665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def downsample(filters, size, apply_batchnorm=True):\n  initializer = tf.random_normal_initializer(0., 0.02)\n\n  result = tf.keras.Sequential()\n  result.add(\n      tf.keras.layers.Conv2D(filters, size, strides=2, padding='same',\n                             kernel_initializer=initializer, use_bias=False))\n\n  if apply_batchnorm:\n    result.add(tf.keras.layers.BatchNormalization())\n\n  result.add(tf.keras.layers.LeakyReLU())\n\n  return result","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:21.866582Z","iopub.execute_input":"2022-06-10T02:53:21.867583Z","iopub.status.idle":"2022-06-10T02:53:21.878421Z","shell.execute_reply.started":"2022-06-10T02:53:21.86754Z","shell.execute_reply":"2022-06-10T02:53:21.877132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"down_model = downsample(3, 4)\ndown_result = down_model(tf.expand_dims(inp, 0))\nprint (down_result.shape)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:25.240354Z","iopub.execute_input":"2022-06-10T02:53:25.240969Z","iopub.status.idle":"2022-06-10T02:53:31.258917Z","shell.execute_reply.started":"2022-06-10T02:53:25.240934Z","shell.execute_reply":"2022-06-10T02:53:31.258042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def upsample(filters, size, apply_dropout=False):\n  initializer = tf.random_normal_initializer(0., 0.02)\n\n  result = tf.keras.Sequential()\n  result.add(\n    tf.keras.layers.Conv2DTranspose(filters, size, strides=2,\n                                    padding='same',\n                                    kernel_initializer=initializer,\n                                    use_bias=False))\n\n  result.add(tf.keras.layers.BatchNormalization())\n\n  if apply_dropout:\n      result.add(tf.keras.layers.Dropout(0.5))\n\n  result.add(tf.keras.layers.ReLU())\n\n  return result","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:36.76328Z","iopub.execute_input":"2022-06-10T02:53:36.763688Z","iopub.status.idle":"2022-06-10T02:53:36.770111Z","shell.execute_reply.started":"2022-06-10T02:53:36.763651Z","shell.execute_reply":"2022-06-10T02:53:36.769353Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"up_model = upsample(3, 4)\nup_result = up_model(down_result)\nprint (up_result.shape)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:42.407452Z","iopub.execute_input":"2022-06-10T02:53:42.408149Z","iopub.status.idle":"2022-06-10T02:53:42.871504Z","shell.execute_reply.started":"2022-06-10T02:53:42.408115Z","shell.execute_reply":"2022-06-10T02:53:42.870527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def Generator():\n  inputs = tf.keras.layers.Input(shape=[256, 256, 3])\n\n  down_stack = [\n    downsample(64, 4, apply_batchnorm=False),  # (batch_size, 128, 128, 64)\n    downsample(128, 4),  # (batch_size, 64, 64, 128)\n    downsample(256, 4),  # (batch_size, 32, 32, 256)\n    downsample(512, 4),  # (batch_size, 16, 16, 512)\n    downsample(512, 4),  # (batch_size, 8, 8, 512)\n    downsample(512, 4),  # (batch_size, 4, 4, 512)\n    downsample(512, 4),  # (batch_size, 2, 2, 512)\n    downsample(512, 4),  # (batch_size, 1, 1, 512)\n  ]\n\n  up_stack = [\n    upsample(512, 4, apply_dropout=True),  # (batch_size, 2, 2, 1024)\n    upsample(512, 4, apply_dropout=True),  # (batch_size, 4, 4, 1024)\n    upsample(512, 4, apply_dropout=True),  # (batch_size, 8, 8, 1024)\n    upsample(512, 4),  # (batch_size, 16, 16, 1024)\n    upsample(256, 4),  # (batch_size, 32, 32, 512)\n    upsample(128, 4),  # (batch_size, 64, 64, 256)\n    upsample(64, 4),  # (batch_size, 128, 128, 128)\n  ]\n\n  initializer = tf.random_normal_initializer(0., 0.02)\n  last = tf.keras.layers.Conv2DTranspose(OUTPUT_CHANNELS, 4,\n                                         strides=2,\n                                         padding='same',\n                                         kernel_initializer=initializer,\n                                         activation='tanh')  # (batch_size, 256, 256, 3)\n\n  x = inputs\n\n  # Downsampling through the model\n  skips = []\n  for down in down_stack:\n    x = down(x)\n    skips.append(x)\n\n  skips = reversed(skips[:-1])\n\n  # Upsampling and establishing the skip connections\n  for up, skip in zip(up_stack, skips):\n    x = up(x)\n    x = tf.keras.layers.Concatenate()([x, skip])\n\n  x = last(x)\n\n  return tf.keras.Model(inputs=inputs, outputs=x)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:46.480146Z","iopub.execute_input":"2022-06-10T02:53:46.480512Z","iopub.status.idle":"2022-06-10T02:53:46.49436Z","shell.execute_reply.started":"2022-06-10T02:53:46.480483Z","shell.execute_reply":"2022-06-10T02:53:46.493642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"generator = Generator()","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:50.647302Z","iopub.execute_input":"2022-06-10T02:53:50.648086Z","iopub.status.idle":"2022-06-10T02:53:51.254352Z","shell.execute_reply.started":"2022-06-10T02:53:50.64805Z","shell.execute_reply":"2022-06-10T02:53:51.253525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"LAMBDA = 100\nloss_object = tf.keras.losses.BinaryCrossentropy(from_logits=True)\n\ndef generator_loss(disc_generated_output, gen_output, target):\n  gan_loss = loss_object(tf.ones_like(disc_generated_output), disc_generated_output)\n\n  # Mean absolute error\n  l1_loss = tf.reduce_mean(tf.abs(target - gen_output))\n\n  total_gen_loss = gan_loss + (LAMBDA * l1_loss)\n\n  return total_gen_loss, gan_loss, l1_loss","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:53.639519Z","iopub.execute_input":"2022-06-10T02:53:53.640113Z","iopub.status.idle":"2022-06-10T02:53:53.647053Z","shell.execute_reply.started":"2022-06-10T02:53:53.640078Z","shell.execute_reply":"2022-06-10T02:53:53.646227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def Discriminator():\n  initializer = tf.random_normal_initializer(0., 0.02)\n\n  inp = tf.keras.layers.Input(shape=[256, 256, 3], name='input_image')\n  tar = tf.keras.layers.Input(shape=[256, 256, 1], name='target_image')\n\n  x = tf.keras.layers.concatenate([inp, tar])  # (batch_size, 256, 256, channels*2)\n\n  down1 = downsample(64, 4, False)(x)  # (batch_size, 128, 128, 64)\n  down2 = downsample(128, 4)(down1)  # (batch_size, 64, 64, 128)\n  down3 = downsample(256, 4)(down2)  # (batch_size, 32, 32, 256)\n\n  zero_pad1 = tf.keras.layers.ZeroPadding2D()(down3)  # (batch_size, 34, 34, 256)\n  conv = tf.keras.layers.Conv2D(512, 4, strides=1,\n                                kernel_initializer=initializer,\n                                use_bias=False)(zero_pad1)  # (batch_size, 31, 31, 512)\n\n  batchnorm1 = tf.keras.layers.BatchNormalization()(conv)\n#   batchnorm1 = conv\n  leaky_relu = tf.keras.layers.LeakyReLU()(batchnorm1)\n\n  zero_pad2 = tf.keras.layers.ZeroPadding2D()(leaky_relu)  # (batch_size, 33, 33, 512)\n\n  last = tf.keras.layers.Conv2D(1, 4, strides=1,\n                                kernel_initializer=initializer)(zero_pad2)  # (batch_size, 30, 30, 1)\n\n  return tf.keras.Model(inputs=[inp, tar], outputs=last)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:53:59.98551Z","iopub.execute_input":"2022-06-10T02:53:59.986215Z","iopub.status.idle":"2022-06-10T02:53:59.996213Z","shell.execute_reply.started":"2022-06-10T02:53:59.986184Z","shell.execute_reply":"2022-06-10T02:53:59.995247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"discriminator = Discriminator()","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:54:02.783639Z","iopub.execute_input":"2022-06-10T02:54:02.784295Z","iopub.status.idle":"2022-06-10T02:54:02.879767Z","shell.execute_reply.started":"2022-06-10T02:54:02.784259Z","shell.execute_reply":"2022-06-10T02:54:02.879026Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def discriminator_loss(disc_real_output, disc_generated_output):\n  real_loss = loss_object(tf.ones_like(disc_real_output), disc_real_output)\n\n  generated_loss = loss_object(tf.zeros_like(disc_generated_output), disc_generated_output)\n\n  total_disc_loss = real_loss + generated_loss\n\n  return total_disc_loss","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:54:04.975737Z","iopub.execute_input":"2022-06-10T02:54:04.976088Z","iopub.status.idle":"2022-06-10T02:54:04.981185Z","shell.execute_reply.started":"2022-06-10T02:54:04.97606Z","shell.execute_reply":"2022-06-10T02:54:04.980375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\ndiscriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:54:07.720665Z","iopub.execute_input":"2022-06-10T02:54:07.721492Z","iopub.status.idle":"2022-06-10T02:54:07.726965Z","shell.execute_reply.started":"2022-06-10T02:54:07.721444Z","shell.execute_reply":"2022-06-10T02:54:07.726238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"checkpoint_dir = './training_checkpoints'\ncheckpoint_prefix = os.path.join(checkpoint_dir, \"ckpt\")\ncheckpoint = tf.train.Checkpoint(generator_optimizer=generator_optimizer,\n                                 discriminator_optimizer=discriminator_optimizer,\n                                 generator=generator,\n                                 discriminator=discriminator)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:54:09.899856Z","iopub.execute_input":"2022-06-10T02:54:09.900218Z","iopub.status.idle":"2022-06-10T02:54:09.907137Z","shell.execute_reply.started":"2022-06-10T02:54:09.900186Z","shell.execute_reply":"2022-06-10T02:54:09.906367Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def generate_images(model, test_input, tar):\n  prediction = model(test_input, training=True)\n  plt.figure(figsize=(15, 15))\n\n  display_list = [test_input[0], tar[0], prediction[0]]\n  title = ['Input Image', 'Ground Truth', 'Predicted Image']\n\n  for i in range(3):\n    print(display_list[i].shape)\n    plt.subplot(1, 3, i+1)\n    plt.title(title[i])\n    # Getting the pixel values in the [0, 1] range to plot.\n    plt.imshow(display_list[i] * 0.5 + 0.5)\n    plt.axis('off')\n  plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:54:13.284772Z","iopub.execute_input":"2022-06-10T02:54:13.28554Z","iopub.status.idle":"2022-06-10T02:54:13.292235Z","shell.execute_reply.started":"2022-06-10T02:54:13.285501Z","shell.execute_reply":"2022-06-10T02:54:13.291412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for example_input, example_target in train_dataset.take(1):\n  generate_images(generator, example_input, example_target)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:54:16.633088Z","iopub.execute_input":"2022-06-10T02:54:16.633745Z","iopub.status.idle":"2022-06-10T02:54:22.200119Z","shell.execute_reply.started":"2022-06-10T02:54:16.63371Z","shell.execute_reply":"2022-06-10T02:54:22.199265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"log_dir=\"logs/\"\n\nsummary_writer = tf.summary.create_file_writer(\n  log_dir + \"fit/\" + datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:54:56.646726Z","iopub.execute_input":"2022-06-10T02:54:56.647705Z","iopub.status.idle":"2022-06-10T02:54:56.656865Z","shell.execute_reply.started":"2022-06-10T02:54:56.647657Z","shell.execute_reply":"2022-06-10T02:54:56.655954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"@tf.function\ndef train_step(input_image, target, step):\n  with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:\n    gen_output = generator(input_image, training=True)\n\n    disc_real_output = discriminator([input_image, target], training=True)\n    disc_generated_output = discriminator([input_image, gen_output], training=True)\n\n    gen_total_loss, gen_gan_loss, gen_l1_loss = generator_loss(disc_generated_output, gen_output, target)\n    disc_loss = discriminator_loss(disc_real_output, disc_generated_output)\n\n  generator_gradients = gen_tape.gradient(gen_total_loss,\n                                          generator.trainable_variables)\n  discriminator_gradients = disc_tape.gradient(disc_loss,\n                                               discriminator.trainable_variables)\n\n  generator_optimizer.apply_gradients(zip(generator_gradients,\n                                          generator.trainable_variables))\n  discriminator_optimizer.apply_gradients(zip(discriminator_gradients,\n                                              discriminator.trainable_variables))\n\n  with summary_writer.as_default():\n    tf.summary.scalar('gen_total_loss', gen_total_loss, step=step//1000)\n    tf.summary.scalar('gen_gan_loss', gen_gan_loss, step=step//1000)\n    tf.summary.scalar('gen_l1_loss', gen_l1_loss, step=step//1000)\n    tf.summary.scalar('disc_loss', disc_loss, step=step//1000)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:54:59.028549Z","iopub.execute_input":"2022-06-10T02:54:59.029268Z","iopub.status.idle":"2022-06-10T02:54:59.045075Z","shell.execute_reply.started":"2022-06-10T02:54:59.029226Z","shell.execute_reply":"2022-06-10T02:54:59.044346Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def fit(train_ds, test_ds, steps):\n  example_input, example_target = next(iter(test_ds.take(1)))\n  start = time.time()\n\n  for step, (input_image, target) in train_ds.repeat().take(steps).enumerate():\n    if (step) % 1000 == 0:\n      display.clear_output(wait=True)\n\n      if step != 0:\n        print(f'Time taken for 1000 steps: {time.time()-start:.2f} sec\\n')\n\n      start = time.time()\n\n      generate_images(generator, example_input, example_target)\n      print(f\"Step: {step//1000}k\")\n\n    train_step(input_image, target, step)\n\n    # Training step\n    if (step+1) % 10 == 0:\n      print('.', end='', flush=True)\n\n\n    # Save (checkpoint) the model every 5k steps\n    if (step + 1) % 5000 == 0:\n      checkpoint.save(file_prefix=checkpoint_prefix)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:55:02.697068Z","iopub.execute_input":"2022-06-10T02:55:02.697451Z","iopub.status.idle":"2022-06-10T02:55:02.705011Z","shell.execute_reply.started":"2022-06-10T02:55:02.697419Z","shell.execute_reply":"2022-06-10T02:55:02.704163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fit(train_dataset, train_dataset, steps=100000)","metadata":{"execution":{"iopub.status.busy":"2022-06-10T02:55:06.364116Z","iopub.execute_input":"2022-06-10T02:55:06.364489Z","iopub.status.idle":"2022-06-10T04:29:02.480212Z","shell.execute_reply.started":"2022-06-10T02:55:06.364457Z","shell.execute_reply":"2022-06-10T04:29:02.47936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TEST_DIR = BASE_DIR+\"/test/test/\"\nOUTPUT_DIR = \"/kaggle/working/results\"\n\ntry:\n    os.mkdir(OUTPUT_DIR)\nexcept Exception:\n    pass","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_img_paths = sorted(glob.glob(TEST_DIR+\"*.png\"))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i,full_pth in enumerate(final_img_paths):\n    input_image, real_image = load(full_pth, full_pth)\n    input_image, real_image = resize(input_image/255.0, real_image, 256, 256)\n    input_image = tf.expand_dims(input_image, axis=0)\n    \n    res = generator(input_image, training=True)[0].cpu().numpy()\n    res = (res*255)\n    res = cv2.resize(res, (400,400))\n    \n    filename = full_pth.split('/')[-1]\n    cv2.imwrite(f'{OUTPUT_DIR}/{filename}', res)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n    index_list = random.sample(range(0, len(final_img_paths)), 3)\nexcept:\n    print('Sample size exceeded population size.')\n    sys.exit(1)\n\nfig, ax = plt.subplots(nrows=1, ncols=6, figsize=(30, 16))\nfor i, index in enumerate(index_list):\n    filename = final_img_paths[index].split('/')[-1]\n    input_img = cv2.imread(final_img_paths[index])\n    output_img = cv2.imread(f'{OUTPUT_DIR}/{filename}')\n    \n    ax[2*i].set_title(\"Input Image\", size=20)\n    ax[2*i].imshow(input_img)\n    ax[2*i+1].set_title(\"Result Image\", size=20)\n    ax[2*i+1].imshow(output_img)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Community\n\nAll official communication will be done through Slack.\n[Join in the discussion](https://join.slack.com/t/denoisingshabby/shared_invite/zt-160i3x0c7-3mGB81~5M0mt2t0zcq2N_Q) happening right now on Slack to see how others are approaching a solution and to access other opportunities to learn!","metadata":{}}]}