{"cells":[{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{},"cell_type":"markdown","source":"Credit : The below tutorial has been adapted from the following resources - official documentation of TensorFlow Privacy , works of Professor Enrique Barra, PhD , Fabian Garcia Pastor , kaggle notebook of Yirun Zhang . The optimizer code is taken from the work of Mayank Shah\n"},{"metadata":{},"cell_type":"markdown","source":"# Privacy in the BigData World\n\nAs much as predictive powers of Big Data revolutionizes the way the businesses across the globe operate , it also poses major security threats and privacy concerns . Consider the famous incident of a pregnant teenager being sent coupons for baby products based on her historical buying data . The increasing use of algorithms for predictions increases the risk that private information which was not willingly disclosed is nevertheless extracted.\n\n\n\n**Pre-emption** is one of the big concerns surrounding big data analytics which involves reducing the person’s range of future actions . Consider an example of a company analyzing its employee behavior to analyze who will be with them for longer term and who is likely to quit soon . And if the company decides to offer educational training and services to the employees who were predicted to stay with them for a longer duration , it could have huge implications . Let's consider the implications of pre-emption in more complex scenarios like the judicial system where at the moment only convicts are imposed punishments . Decisions like preventing the individuals who have higher probability to commit crime from traveling can prevent crime before they occur. Both prediction and pre-emption as powerful as they seem can lead to several ethical concerns .\n\n🎯 What kind of prediction is ethically acceptable?\n\n🎯 When is a prediction profound or strong enough to justify consequences? \n\n🎯 What restrictions on individuals can be justified based on predictions? \n\n🎯 Which information should be allowed as a basis for predictions?\n\n## Differential Privacy\n\n\nDifferential privacy is a guarantee to protect data and addresses the risk of information about a particular value being released when queries are sent to datasets. Differential privacy works particularly well in Big Data systems and it makes inference and tracking attacks less likely. Consider the scenario of an organization interested in using its user database for research purposes . Since the database contains sensitive data about users , the organization has to anonymize its user data before using it for research .\n
\n\nMajor application of differential privacy is in healthcare where there is a trade-off between protecting sensitive information about patients and mining useful information from datasets in order to determine health trends.Consider an example where a hospital may have a database of patient records and each record contains a binary value indicating whether or not the patient has a particular disease. Such information is tracked to keep record of the total number of patients with the disease , but there could also be a possibility that the hospital or third parties are interested to investigate correlations between the disease and age or gender and the disease or any other factor.Individual patient specific data has to be hidden in this case as patients may not prefer others getting to know they have a disease . So when the above data is used for research , it has to be ensured that the data is analysed in a meaningful way without violation of privacy. \n\n\n### Advantages of differential privacy :\n\n📌 Limits the amount of information that any analyst can learn about an individual \n\n📌 Protects individuals' privacy. \n\n📌 It also makes inference and tracking attacks less likely as a great deal of complexity is needed to infer information or track individuals .\n\n### Differential Privacy Methods:\n\n\nDifferential privacy uses suited algorithms that add a sufficient amount of noise to a data set in order to guarantee that nothing specific is being revealed about an individual from the data. The implications of adding or removing a single individual data point is relatively smaller compared to the noise being added . This mechanism does not cause any significant change to the overall outcome of an analysis.\n
\n\n\n\n\n**Laplace mechanism** is applied to anonymize statistical aggregates like average . Random number is added to the average ( ie) noise gets added to the laplace distribution ) . Consider the scenario where we divide the population into groups based on age , social and physical aspects and the average number of people having disease in each sub group is released . This could lead to the attacker learning a lot about the subgroups and can even result in attackers identifying the individuals in each subgroup. \n\n**A randomized response procedure** that helps to protect the privacy of survey participants. Consider a scenario where for each survey response , two coins are flipped using a random number generator .The results of flipped coins are decoded as follows .\n\n\n \nThis process has distorted the survey results in a statistical way and it is impossible to derive information about the response of individual participants. Although it has to be noted that some statistical calculations can still help determine actual survey results and the approach only works for a larger group of participants .\n\n**The exponential mechanism** uses a quality score to rank the output based on its representation of actual input data. Every possible database entry gets a score based on its likelihood derived from the input data and outputs which have higher scores with a higher probability are chosen. The synthetic dataset generated based on the above mechanism allows for the same statistical conclusions as the original database while preserving privacy as it does not provide real data and hence individual specific information cannot be inferred .\n"},{"metadata":{},"cell_type":"markdown","source":"# TensorFlow Privacy :\n\n\nTensorFlow Privacy wraps an existing TensorFlow optimizer to create a variant that implements Differentially private stochastic gradient descent (DP-SGD) .(DP-SGD) modifies the gradients used in stochastic gradient descent (SGD) and the models trained with DP-SGD provide provable differential privacy guarantees for their input data.\n
\n\nThe two modifications made to SGD are below. \n\n🎯 The sensitivity of each gradient is bounded by clipping each gradient computed on each training point . This limits how much each individual training point sampled in a minibatch can influence gradient computations and the resulting updates applied to model parameters.\n\n🎯 Random noise is sample and added to clipped gradients which makes it statistically impossible to know whether or not a particular data point was included in the training dataset \n \nDP-SGD has three privacy-specific hyperparameters and one existing hyperparameter that requires tuning\n\n📌l2_norm_clip (float) - The maximum Euclidean (L2) norm of each gradient that is applied to update model parameters. This hyperparameter is used to bound the optimizer's sensitivity to individual training points.\n\n📌noise_multiplier (float) - The amount of noise sampled and added to gradients during training. Generally, more noise results in better privacy (often, but not necessarily, at the expense of lower utility).\n\n📌microbatches (int) - Each batch of data is split in smaller units called microbatches. By default, each microbatch should contain a single training example. This allows us to clip gradients on a per-example basis rather than after they have been averaged across the minibatch. This in turn decreases the (negative) effect of clipping on signal found in the gradient and typically maximizes utility.\n\n📌learning_rate (float) - This hyperparameter already exists in vanilla SGD. The higher the learning rate, the more each update matters. If the updates are noisy (such as when the additive noise is large compared to the clipping threshold), a low learning rate may help the training procedure converge.\n"},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"!pip install tensorflow_privacy\n\nfrom tensorflow_privacy.privacy.analysis import compute_dp_sgd_privacy\nfrom tensorflow_privacy.privacy.optimizers.dp_optimizer import DPGradientDescentGaussianOptimizer","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Credit : The code below is an adapted from Yirun Zhang Kaggle notebook"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\")\n\nimport gc\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\nimport tensorflow.keras.backend as K\nimport tensorflow.keras.layers as L\nimport tensorflow.keras.models as M\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, ModelCheckpoint, EarlyStopping\nimport tensorflow_addons as tfa\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import log_loss\nfrom hyperopt import hp, fmin, tpe, Trials\nfrom hyperopt.pyll.base import scope\nfrom tqdm.notebook import tqdm\n\nprint('Tensorflow version:', tf.__version__)\nAUTO = tf.data.experimental.AUTOTUNE","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Detect hardware, return appropriate distribution strategy\ntry:\n tpu = tf.distribute.cluster_resolver.TPUClusterResolver() # TPU detection. No parameters necessary if TPU_NAME environment variable is set. On Kaggle this is always the case.\n print('Running on TPU ', tpu.master())\nexcept ValueError:\n tpu = None\n\nif tpu:\n tf.config.experimental_connect_to_cluster(tpu)\n tf.tpu.experimental.initialize_tpu_system(tpu)\n strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n strategy = tf.distribute.get_strategy() # default distribution strategy in Tensorflow. Works on CPU and single GPU.\n\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"MIXED_PRECISION = False\nXLA_ACCELERATE = True\n\nif MIXED_PRECISION:\n from tensorflow.keras.mixed_precision import experimental as mixed_precision\n if tpu: policy = tf.keras.mixed_precision.experimental.Policy('mixed_bfloat16')\n else: policy = tf.keras.mixed_precision.experimental.Policy('mixed_float16')\n mixed_precision.set_policy(policy)\n print('Mixed precision enabled')\n\nif XLA_ACCELERATE:\n tf.config.optimizer.set_jit(True)\n print('Accelerated Linear Algebra enabled')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"train_features = pd.read_csv('../input/lish-moa/train_features.csv')\ntrain_targets = pd.read_csv('../input/lish-moa/train_targets_scored.csv')\ntest_features = pd.read_csv('../input/lish-moa/test_features.csv')\n\nss = pd.read_csv('../input/lish-moa/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def preprocess(df):\n df.loc[:, 'cp_type'] = df.loc[:, 'cp_type'].map({'trt_cp': 0, 'ctl_vehicle': 1})\n df.loc[:, 'cp_dose'] = df.loc[:, 'cp_dose'].map({'D1': 0, 'D2': 1})\n del df['sig_id']\n return df\n\ntrain = preprocess(train_features)\ntest = preprocess(test_features)\n\ndel train_targets['sig_id']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"top_feats = [ 0, 1, 2, 3, 5, 6, 8, 9, 10, 11, 12, 14, 15,\n 16, 18, 19, 20, 21, 23, 24, 25, 27, 28, 29, 30, 31,\n 32, 33, 34, 35, 36, 37, 39, 40, 41, 42, 44, 45, 46,\n 48, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61,\n 63, 64, 65, 66, 68, 69, 70, 71, 72, 73, 74, 75, 76,\n 78, 79, 80, 81, 82, 83, 84, 86, 87, 88, 89, 90, 92,\n 93, 94, 95, 96, 97, 99, 100, 101, 103, 104, 105, 106, 107,\n 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120,\n 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 132, 133, 134,\n 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147,\n 149, 150, 151, 152, 153, 154, 155, 157, 159, 160, 161, 163, 164,\n 165, 166, 167, 168, 169, 170, 172, 173, 175, 176, 177, 178, 180,\n 181, 182, 183, 184, 186, 187, 188, 189, 190, 191, 192, 193, 195,\n 197, 198, 199, 202, 203, 205, 206, 208, 209, 210, 211, 212, 213,\n 214, 215, 218, 219, 220, 221, 222, 224, 225, 227, 228, 229, 230,\n 231, 232, 233, 234, 236, 238, 239, 240, 241, 242, 243, 244, 245,\n 246, 248, 249, 250, 251, 253, 254, 255, 256, 257, 258, 259, 260,\n 261, 263, 265, 266, 268, 270, 271, 272, 273, 275, 276, 277, 279,\n 282, 283, 286, 287, 288, 289, 290, 294, 295, 296, 297, 299, 300,\n 301, 302, 303, 304, 305, 306, 308, 309, 310, 311, 312, 313, 315,\n 316, 317, 320, 321, 322, 324, 325, 326, 327, 328, 329, 330, 331,\n 332, 333, 334, 335, 338, 339, 340, 341, 343, 344, 345, 346, 347,\n 349, 350, 351, 352, 353, 355, 356, 357, 358, 359, 360, 361, 362,\n 363, 364, 365, 366, 368, 369, 370, 371, 372, 374, 375, 376, 377,\n 378, 379, 380, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391,\n 392, 393, 394, 395, 397, 398, 399, 400, 401, 403, 405, 406, 407,\n 408, 410, 411, 412, 413, 414, 415, 417, 418, 419, 420, 421, 422,\n 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435,\n 436, 437, 438, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450,\n 452, 453, 454, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465,\n 466, 468, 469, 471, 472, 473, 474, 475, 476, 477, 478, 479, 482,\n 483, 485, 486, 487, 488, 489, 491, 492, 494, 495, 496, 500, 501,\n 502, 503, 505, 506, 507, 509, 510, 511, 512, 513, 514, 516, 517,\n 518, 519, 521, 523, 525, 526, 527, 528, 529, 530, 531, 532, 533,\n 534, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547,\n 549, 550, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563,\n 564, 565, 566, 567, 569, 570, 571, 572, 573, 574, 575, 577, 580,\n 581, 582, 583, 586, 587, 590, 591, 592, 593, 595, 596, 597, 598,\n 599, 600, 601, 602, 603, 605, 607, 608, 609, 611, 612, 613, 614,\n 615, 616, 617, 619, 622, 623, 625, 627, 630, 631, 632, 633, 634,\n 635, 637, 638, 639, 642, 643, 644, 645, 646, 647, 649, 650, 651,\n 652, 654, 655, 658, 659, 660, 661, 662, 663, 664, 666, 667, 668,\n 669, 670, 672, 674, 675, 676, 677, 678, 680, 681, 682, 684, 685,\n 686, 687, 688, 689, 691, 692, 694, 695, 696, 697, 699, 700, 701,\n 702, 703, 704, 705, 707, 708, 709, 711, 712, 713, 714, 715, 716,\n 717, 723, 725, 727, 728, 729, 730, 731, 732, 734, 736, 737, 738,\n 739, 740, 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751,\n 752, 753, 754, 755, 756, 758, 759, 760, 761, 762, 763, 764, 765,\n 766, 767, 769, 770, 771, 772, 774, 775, 780, 781, 782, 783, 784,\n 785, 787, 788, 790, 793, 795, 797, 799, 800, 801, 805, 808, 809,\n 811, 812, 813, 816, 819, 820, 821, 822, 823, 825, 826, 827, 829,\n 831, 832, 833, 834, 835, 837, 838, 839, 840, 841, 842, 844, 845,\n 846, 847, 848, 850, 851, 852, 854, 855, 856, 858, 860, 861, 862,\n 864, 867, 868, 870, 871, 873, 874]\n\nprint(len(top_feats))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Credit :\nThe author of the optimizer code is Mayank Shah - https://github.com/mayankshah1607"},{"metadata":{},"cell_type":"markdown","source":"The optimizer code in the official repository isnt compatible with TensorFlow 2.0+ version . The below code by Mayank is a turnaround to avoid the error because of version incompatibility"},{"metadata":{"trusted":true},"cell_type":"code","source":"from absl import logging\nimport collections\n\nfrom tensorflow_privacy.privacy.analysis import privacy_ledger\nfrom tensorflow_privacy.privacy.dp_query import gaussian_query\n\ndef make_optimizer_class(cls):\n \"\"\"Constructs a DP optimizer class from an existing one.\"\"\"\n parent_code = tf.compat.v1.train.Optimizer.compute_gradients.__code__\n child_code = cls.compute_gradients.__code__\n GATE_OP = tf.compat.v1.train.Optimizer.GATE_OP # pylint: disable=invalid-name\n if child_code is not parent_code:\n logging.warning(\n 'WARNING: Calling make_optimizer_class() on class %s that overrides '\n 'method compute_gradients(). Check to ensure that '\n 'make_optimizer_class() does not interfere with overridden version.',\n cls.__name__)\n\n class DPOptimizerClass(cls):\n \"\"\"Differentially private subclass of given class cls.\"\"\"\n\n _GlobalState = collections.namedtuple(\n '_GlobalState', ['l2_norm_clip', 'stddev'])\n \n def __init__(\n self,\n dp_sum_query,\n num_microbatches=None,\n unroll_microbatches=False,\n *args, # pylint: disable=keyword-arg-before-vararg, g-doc-args\n **kwargs):\n \"\"\"Initialize the DPOptimizerClass.\n\n Args:\n dp_sum_query: DPQuery object, specifying differential privacy\n mechanism to use.\n num_microbatches: How many microbatches into which the minibatch is\n split. If None, will default to the size of the minibatch, and\n per-example gradients will be computed.\n unroll_microbatches: If true, processes microbatches within a Python\n loop instead of a tf.while_loop. Can be used if using a tf.while_loop\n raises an exception.\n \"\"\"\n super(DPOptimizerClass, self).__init__(*args, **kwargs)\n self._dp_sum_query = dp_sum_query\n self._num_microbatches = num_microbatches\n self._global_state = self._dp_sum_query.initial_global_state()\n # TODO(b/122613513): Set unroll_microbatches=True to avoid this bug.\n # Beware: When num_microbatches is large (>100), enabling this parameter\n # may cause an OOM error.\n self._unroll_microbatches = unroll_microbatches\n\n def compute_gradients(self,\n loss,\n var_list,\n gate_gradients=GATE_OP,\n aggregation_method=None,\n colocate_gradients_with_ops=False,\n grad_loss=None,\n gradient_tape=None,\n curr_noise_mult=0,\n curr_norm_clip=1):\n\n self._dp_sum_query = gaussian_query.GaussianSumQuery(curr_norm_clip, \n curr_norm_clip*curr_noise_mult)\n self._global_state = self._dp_sum_query.make_global_state(curr_norm_clip, \n curr_norm_clip*curr_noise_mult)\n \n\n # TF is running in Eager mode, check we received a vanilla tape.\n if not gradient_tape:\n raise ValueError('When in Eager mode, a tape needs to be passed.')\n\n vector_loss = loss()\n if self._num_microbatches is None:\n self._num_microbatches = tf.shape(input=vector_loss)[0]\n sample_state = self._dp_sum_query.initial_sample_state(var_list)\n microbatches_losses = tf.reshape(vector_loss, [self._num_microbatches, -1])\n sample_params = (self._dp_sum_query.derive_sample_params(self._global_state))\n\n def process_microbatch(i, sample_state):\n \"\"\"Process one microbatch (record) with privacy helper.\"\"\"\n microbatch_loss = tf.reduce_mean(input_tensor=tf.gather(microbatches_losses, [i]))\n grads = gradient_tape.gradient(microbatch_loss, var_list)\n sample_state = self._dp_sum_query.accumulate_record(sample_params, sample_state, grads)\n return sample_state\n \n for idx in range(self._num_microbatches):\n sample_state = process_microbatch(idx, sample_state)\n\n if curr_noise_mult > 0:\n grad_sums, self._global_state = (self._dp_sum_query.get_noised_result(sample_state, self._global_state))\n else:\n grad_sums = sample_state\n\n def normalize(v):\n return v / tf.cast(self._num_microbatches, tf.float32)\n\n final_grads = tf.nest.map_structure(normalize, grad_sums)\n grads_and_vars = final_grads#list(zip(final_grads, var_list))\n \n return grads_and_vars\n\n return DPOptimizerClass\n\n\ndef make_gaussian_optimizer_class(cls):\n \"\"\"Constructs a DP optimizer with Gaussian averaging of updates.\"\"\"\n\n class DPGaussianOptimizerClass(make_optimizer_class(cls)):\n \"\"\"DP subclass of given class cls using Gaussian averaging.\"\"\"\n\n def __init__(\n self,\n l2_norm_clip,\n noise_multiplier,\n num_microbatches=None,\n ledger=None,\n unroll_microbatches=False,\n *args, # pylint: disable=keyword-arg-before-vararg\n **kwargs):\n dp_sum_query = gaussian_query.GaussianSumQuery(\n l2_norm_clip, l2_norm_clip * noise_multiplier)\n\n if ledger:\n dp_sum_query = privacy_ledger.QueryWithLedger(dp_sum_query,\n ledger=ledger)\n\n super(DPGaussianOptimizerClass, self).__init__(\n dp_sum_query,\n num_microbatches,\n unroll_microbatches,\n *args,\n **kwargs)\n\n @property\n def ledger(self):\n return self._dp_sum_query.ledger\n\n return DPGaussianOptimizerClass","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"GradientDescentOptimizer = tf.compat.v1.train.GradientDescentOptimizer\nDPGradientDescentGaussianOptimizer_NEW = make_gaussian_optimizer_class(GradientDescentOptimizer)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"l2_norm_clip = 1.5\nnoise_multiplier = 1.3\nnum_microbatches = 64\nlearning_rate = 1e-3\n\noptimizer = DPGradientDescentGaussianOptimizer_NEW(\n l2_norm_clip=l2_norm_clip,\n noise_multiplier=noise_multiplier,\n num_microbatches=num_microbatches,\n learning_rate=learning_rate)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope(): \n model = tf.keras.Sequential([\n tf.keras.layers.Input(len(top_feats)),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dropout(0.2),\n tfa.layers.WeightNormalization(tf.keras.layers.Dense(2048, activation=\"relu\")),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dropout(0.5),\n tfa.layers.WeightNormalization(tf.keras.layers.Dense(1048, activation=\"relu\")),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dropout(0.5),\n tfa.layers.WeightNormalization(tf.keras.layers.Dense(206, activation=\"sigmoid\"))\n ])\n model.compile(optimizer=optimizer,loss=keras.losses.BinaryCrossentropy())\n ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Measure the differential privacy guarantee\n\n{Quote from official documentation below }\n\nPerform a privacy analysis to measure the DP guarantee achieved by a training algorithm. Knowing the level of DP achieved enables the objective comparison of two training runs to determine which of the two is more privacy-preserving. At a high level, the privacy analysis measures how much a potential adversary can improve their guess about properties of any individual training point by observing the outcome of our training procedure (e.g., model updates and parameters).\n\nThis guarantee is sometimes referred to as the **privacy budget**. A lower privacy budget bounds more tightly an adversary's ability to improve their guess. This ensures a stronger privacy guarantee. Intuitively, this is because it is harder for a single training point to affect the outcome of learning: for instance, the information contained in the training point cannot be memorized by the ML algorithm and the privacy of the individual who contributed this training point to the dataset is preserved.\nThe privacy analysis here is performed in the framework of Rényi Differential Privacy (RDP) -(research paper)\n \nTwo metrics are used to express the DP guarantee of an ML algorithm:\n\n🎯Delta () - Bounds the probability of the privacy guarantee not holding. A rule of thumb is to set it to be less than the inverse of the size of the training dataset.\n\n🎯Epsilon () - This is the privacy budget. It measures the strength of the privacy guarantee by bounding how much the probability of a particular model output can vary by including (or excluding) a single training point and a smaller value implies a better privacy guarantee.\n\nTensorflow Privacy provides a tool, compute_dp_sgd_privacy.py, to compute the value of given a fixed value of and the following hyperparameters from the training process:\n \n📌The total number of points in the training data, n.\n\n📌The batch_size.\n\n📌The noise_multiplier.\n\n📌The number of epochs of training.\n"},{"metadata":{},"cell_type":"markdown","source":"References :\n\nhttps://github.com/tensorflow/privacy\n\nhttps://github.com/mayankshah1607\n\nhttps://www.kaggle.com/gogo827jz/hyperparameter-tuning-for-neural-network-on-tpu"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}