{
  "id": 330377,
  "title": "💡 Tips for Learning Deep Learning Faster",
  "url": "/competitions/digit-recognizer/discussion/330377",
  "author_name": "The Devastator",
  "post_date": "2022-06-12T01:02:39.851000",
  "votes": 1,
  "comment_count": 0,
  "views": null,
  "content": "<h1>💡 Tips for Learning Deep Learning Faster</h1>\n<h4>Welcome to the area you are interested in! 🎉 🎊</h4>\n<p>Building a NN is the easy and hard part, but the understanding is what's difficult. When you want to learn something, you start, precisely, with the topic that you are interested in.</p>\n<p>First Tip: Work on a problem you <strong>really</strong> care about, this way you can sustain the motivation needed to progress.</p>\n<h4>General Knowledge</h4>\n<ul>\n<li><p>Get ready to obtain broad knowledge.</p></li>\n<li><p>Read as much as you can - understand the main concepts</p></li>\n<li><p>Know how to code &amp; factorize a problem to code.</p>\n<ul>\n<li>Factorize in smaller parts first</li>\n<li>Math skills - Important.</li></ul></li>\n<li><p>Start with Numpy, python 3, and basic editor/text-editor</p></li>\n<li><p>Plot your data, and associate the concept to your data.</p></li>\n<li><p>Choose your tools </p>\n<ul>\n<li>Pycharm</li>\n<li>Colab</li>\n<li>evn: conda, tensorflow, jupyter, …</li></ul></li>\n</ul>\n<h4>Reading and coding good and CLEAN (simple + short + concise) code</h4>\n<ul>\n<li>Know where all the parameters and objects are.</li>\n<li>Learn from masters; study their thoughts on what your code should include.</li>\n<li>If you don't understand the code, draw some outline/data flow charts.</li>\n<li>Master clean, simple, short, and precise code well before you start to bend over backward in Python with hacking tricks to excessively fancy code ahead of your basic computer science training (for creative, artificial intelligence applications)</li>\n</ul>\n<h4>Know how to read a good paper (Paper Reading Strategies).</h4>\n<ul>\n<li>Comprehend the materials: What are the selling points.</li>\n<li>Understand and figure out concepts the writer is saying.</li>\n<li>Try to implement the paper. (copy it and modify)</li>\n</ul>\n<h4>Study before start coding :</h4>\n<ul>\n<li>Check for correctness during coding.</li>\n<li>Prerequisite about CS training.</li>\n</ul>\n<h4>Start Testing and the details</h4>\n<ul>\n<li>Broken Machines - The errors are very difficult to fix - Understanding how the flow of data works.</li>\n<li>Don't just copy, paste and run code - Understand it first.</li>\n<li>A good experimental design: what to test?</li>\n<li>Output - Plotting results</li>\n<li>Save all results </li>\n</ul>\n<h3>Workflow - Call in for reinforcements</h3>\n<p>Your workflow whether it be a very small one or a much bigger one must be smooth and smart from the start. If you follow the workflow described below and stick to it consistently the chances are great that your project will become more appealing, interesting, and well organized (in addition to more manageable and less prone to bugs).</p>\n<h4>Look for existing code first</h4>\n<ul>\n<li>Python: walk over the code and check for libraries</li>\n<li>Coding at its very center is sharing, and sharing of code is the key point that which Python and other programming languages were founded. When you can't code something on your own and you have no one else to turn to, just go to the Internet and search out real-world examples, applications, and implementations in the popular open-source libraries mentioned below. The engineering of the programs you want is already present. You just need to adapt it to your problem and use it. </li>\n</ul>\n<blockquote>\n  <p>Don't forget to check its license</p>\n</blockquote>\n<ul>\n<li><p>Don´t overcomplicate it - Work in a simple environment that can be upgraded at any time.</p></li>\n<li><p>If you are new to the programming world, consider getting the Python programming language as your primary language.</p>\n<ul>\n<li>Even if you have more experience with another language like Java, Unity, or PHP, mastering Python will simplify your lives in the future for it is both versatile and simple, taking almost little to no time to learn.</li></ul></li>\n<li><p>Care where to grow connections - Kaggle is a great place for this. </p></li>\n<li><p>Choose wisely</p>\n<ul>\n<li>Pick one library at first, and master it before moving further.</li>\n<li>Consider finding the answer and solution to your problem before anything else, review them to see examples for similar problems.</li></ul></li>\n<li><p>Build your own problem </p>\n<ul>\n<li>See with your data how it works</li>\n<li>Prepare your data &lt;=&gt; change the data and learn how to manipulate it.</li></ul></li>\n<li><p>Choose your tools wisely </p>\n<ul>\n<li>Colab, Jupyter Ide</li>\n<li>Visualizing your problem and working on your code</li></ul></li>\n</ul>\n<h4>Dive into a fascinating aspect of NN</h4>\n<ul>\n<li>Understand the graph and what is a neural network all about</li>\n<li>Before Training a model<ul>\n<li>Visualize your data</li>\n<li>Preprocessing in a separate function</li>\n<li>Feature engineering</li></ul></li>\n<li>Use all data you can<ul>\n<li>Visualize the whole data</li>\n<li>Data augmentation (different ways to augment: horizontal flip, or scaling up).</li></ul></li>\n<li>Get your own labels to train<ul>\n<li>Easy to collect: Scraping</li>\n<li>When scraping, check your data for what you collect</li></ul></li>\n<li>Prepare your model <ul>\n<li>Intuitively, or build from known papers</li></ul></li>\n</ul>\n<h4>Make decisions during training</h4>\n<ul>\n<li>Have a logging directory</li>\n<li>Early stopping, regularization, and early stopping<br>\n(it needs to be different to classify different things)</li>\n<li>The loss function</li>\n<li>Understand the semantics of these functions</li>\n</ul>\n<h4>Share the code base</h4>\n<ul>\n<li>Code with config in mind<ul>\n<li>Have a config dictionary in mind</li>\n<li>Have a loading function from the config</li></ul></li>\n<li>Have the directories ready (logging, model files, training files, …)</li>\n<li>Build multiple helper functions for accelerating coding.</li>\n</ul>\n<h4>Visualize your data</h4>\n<ul>\n<li>Plot the data and problem (visualization, image pro)</li>\n<li>Practice the more realistic data to the real world. (not contrived, test your program on real and realistic examples:  images, audio streams…)</li>\n</ul>\n<h4>Preprocessing and processing data</h4>\n<ul>\n<li>Preprocessing stage could be an augmenting stage.</li>\n<li>Train your model on raw data and then on processed data (and possible other data as a whole)</li>\n</ul>\n<h4>Know your network</h4>\n<ul>\n<li>Explain what each layer does</li>\n<li>Plot the graph (observe the inner workings of Deep Learning)</li>\n<li>Learn the semantics of layers of your Network</li>\n</ul>\n<h4>Debugging, Debugging Skills &amp; Debugging Hacks</h4>\n<ul>\n<li>Python comes with a built-in debugger to inspect all your custom code.</li>\n<li>Knowing what's inside the code leads you efficiently through all the debugging processes. Use the experiences, and ideas that you have to solve the problematic part of your code.</li>\n<li>Work in a controlled environment: Plotting, and checking your intermediate outputs if something is going wrong.<ul>\n<li>Check easily where the error is. (rule-of-thumb: simple problem, check top to bottom, complicated problem check bottom-up, )</li>\n<li>Always check the results of your previous experiments: what worked? what didn't? Ask your self \"Why? Why not\".</li></ul></li>\n</ul>\n<h4>Find inspiration easily</h4>\n<ul>\n<li>Understand existing code: Efficiently and effectively utilize the code you bought.</li>\n</ul>\n<h4>The logic behind the code</h4>\n<ul>\n<li>Create your own NN to see how it works.</li>\n<li>Batch Normalization (BN)</li>\n<li>Actual derivation and calculation of gradient descent.</li>\n<li>Regularization: L1 &amp; L2</li>\n</ul>\n<h4>Developing a complete project</h4>\n<ul>\n<li>Record all your parameter in training.<ul>\n<li>Create a mini log version.</li>\n<li>Evaluating the model in different versions.</li>\n<li>Export the version to CVS file so it can be translated into a JSON file, and edit the path to suit your requests.</li>\n<li>Importing data from CSV for better reading/comparison.</li></ul></li>\n<li>Block losses, metrics, notes (gradient descent, …) all functions in separate directories.<ul>\n<li>Easily edit and change whole plans not scripts.</li>\n<li>Any modification, debugging mode.</li>\n<li>Network layers and modules</li></ul></li>\n</ul>\n<h4>Practical Deploying and environments</h4>\n<ul>\n<li>Compute budgets for your projects<ul>\n<li>Project environments (what people stores, how women store their things)<ul>\n<li>Project: different projects require different tools/envs/</li>\n<li>Functionalities: What you know already and what libraries you need to.</li>\n<li>Able to define what you want and expect (how your code works, and how<br>\nthe system is supposed to be)</li>\n<li>How to obtain different compute units (GPU, resources …)</li></ul></li></ul></li>\n<li>Preparing your data - label, and format your dataset.</li>\n<li>Preview your model</li>\n<li>Build the model from config.</li>\n<li>Train the model: pickle the model, ready to test and use.</li>\n</ul>\n<h4>Tools of the trade</h4>\n<ul>\n<li>Learn the library coding convention and software engineering skills in the beginning.</li>\n<li>Debugging skills in Pycharm<ul>\n<li>Debug and track your problem with Python; jupyter notebook or similar libraries.</li>\n<li>Print, count object/memory/number in print(RUNCODE) or other libraries.</li></ul></li>\n<li>Plotting or drawing graphs with Matplotlib - Mplcursors</li>\n<li>Understand the whole algorithm &amp; structures.</li>\n<li>Train in Jupyter Notebook and then apply it on Pycharm.</li>\n</ul>\n<h4>Discipline - Keeping scripts and code clean and understandable</h4>\n<ul>\n<li><p>Each experiment should include unique, dedicated metadata about when, how, and why it was performed.</p>\n<ul>\n<li>Keeping the code, experiments, and their related output organized and shareable across computers.</li>\n<li>An attempt to make the code more deterministic:</li>\n<li>Execute a record and create a directory containing the data and the code of that experiment in addition to the required data of the state of the machine, dependencies, and versions.</li></ul></li>\n<li><p>Ensure you'll be able to discover and reproduce your experiments.</p>\n<ul>\n<li>Document the code, the version of all libraries, and the data of the experiment that should be recovered and what is expected to be obtained.</li>\n<li>Creating a directory tree in a meaningful way.</li>\n<li>Code reads like a book - Organize the subject before you start.</li></ul></li>\n<li><p>Prepare meta-data</p>\n<ul>\n<li>Put all the information related to the criteria in which you employed the data in the stored experiment.</li>\n<li>Prepare the information you will use to recover the experiment.</li>\n<li>Include libraries, archive types, and files important to the experiment.</li></ul></li>\n<li><p>Put the model or indicators in an easy-to-understand place.</p>\n<ul>\n<li>Understand the relationship between the data and the record.</li>\n<li>Keep it readable and understandable.</li>\n<li>Build the record to be well described.</li></ul></li>\n<li><p>Each experiment should be described with the purpose to recover each experiment.</p>\n<ul>\n<li>Put commas inside the file to record the position of the file.</li></ul></li>\n<li><p>Store directories and sub-directories of assets to ease the process of understanding and retrieving the records when needed.</p></li>\n</ul>\n<blockquote>\n  <p>With the abundance of machine learning frameworks and the development of auto-differentiation libraries, code deployment becomes a crucial next step to Get Ready To Start Learning Deep Learning. Let's first look at some \"CNN\"s concepts.</p>\n</blockquote>\n<h4>Coding</h4>\n<h4>Critical Tips on Code Architecture</h4>\n<blockquote>\n  <p>Effective code architecture for facilitating \"Deep Learning Plug and Play\" is a critical aspect of scientific research and utmost productivity.</p>\n</blockquote>\n<h5>Tools of the Trade - Code Architecture</h5>\n<ul>\n<li><p><strong>Follow a video tutorial or a full course</strong></p>\n<ul>\n<li>All of these resources are provided by TensorFlow and PyTorch, the most widely used libraries by practitioners in the academic and industrial look-alike machine learning fields. There is hardly any dissimilarity between modules of these libraries, so going with TensorFlow will do you no difference in practice. These are the resources to go with. They will teach you the ins and outs of how to train deep models and how to build effective ones.</li>\n<li>These resources will make you understand how your computer does work beyond the abstraction level of libraries, for a better understanding of Algorithms. </li></ul></li>\n<li><p><strong>Starting Deep Learning Training by training on low resolution and then go on to higher resolution</strong> </p>\n<ul>\n<li>This has nothing to do with the network itself, it will just save you time if your model won't work on the first run: Time-saving is A MUST when you start. </li></ul></li>\n<li><p><strong>Prepare your data in machine-readable formats</strong> (Numpy arrays, Pytorch tensors, …)</p>\n<ul>\n<li>Cut corners by opting for machine-readable &amp; writable formats, which are JIT-compiled codes of NumPy and so on.</li>\n<li>Understand formatting to the very deep end. It will be important and will save you time. </li></ul></li>\n</ul>\n<h5>Bits and Pieces 🤝</h5>\n<ul>\n<li><strong>Good architectures solve 80% of the software</strong> - <strong>Make sure the code runs smoothly before scaling up the number of samples.</strong> (scaling up the number of samples and starting with huge data sets is not as relevant as a well-designed project.)</li>\n</ul>\n<h4>A Short Concluding Note</h4>\n<ul>\n<li>Good ideas on multiple topics are important. But stick to one at a time when starting out.</li>\n<li>In addition, the ability to invent time, experiment, draw and sketch out something new is the real engine of scientific progress.</li>\n</ul>\n<p>Good Luck!</p>",
  "messages": [
    {
      "id": 1817888,
      "postDate": "2022-06-12T01:02:39.850Z",
      "content": "<h1>💡 Tips for Learning Deep Learning Faster</h1>\n<h4>Welcome to the area you are interested in! 🎉 🎊</h4>\n<p>Building a NN is the easy and hard part, but the understanding is what's difficult. When you want to learn something, you start, precisely, with the topic that you are interested in.</p>\n<p>First Tip: Work on a problem you <strong>really</strong> care about, this way you can sustain the motivation needed to progress.</p>\n<h4>General Knowledge</h4>\n<ul>\n<li><p>Get ready to obtain broad knowledge.</p></li>\n<li><p>Read as much as you can - understand the main concepts</p></li>\n<li><p>Know how to code &amp; factorize a problem to code.</p>\n<ul>\n<li>Factorize in smaller parts first</li>\n<li>Math skills - Important.</li></ul></li>\n<li><p>Start with Numpy, python 3, and basic editor/text-editor</p></li>\n<li><p>Plot your data, and associate the concept to your data.</p></li>\n<li><p>Choose your tools </p>\n<ul>\n<li>Pycharm</li>\n<li>Colab</li>\n<li>evn: conda, tensorflow, jupyter, …</li></ul></li>\n</ul>\n<h4>Reading and coding good and CLEAN (simple + short + concise) code</h4>\n<ul>\n<li>Know where all the parameters and objects are.</li>\n<li>Learn from masters; study their thoughts on what your code should include.</li>\n<li>If you don't understand the code, draw some outline/data flow charts.</li>\n<li>Master clean, simple, short, and precise code well before you start to bend over backward in Python with hacking tricks to excessively fancy code ahead of your basic computer science training (for creative, artificial intelligence applications)</li>\n</ul>\n<h4>Know how to read a good paper (Paper Reading Strategies).</h4>\n<ul>\n<li>Comprehend the materials: What are the selling points.</li>\n<li>Understand and figure out concepts the writer is saying.</li>\n<li>Try to implement the paper. (copy it and modify)</li>\n</ul>\n<h4>Study before start coding :</h4>\n<ul>\n<li>Check for correctness during coding.</li>\n<li>Prerequisite about CS training.</li>\n</ul>\n<h4>Start Testing and the details</h4>\n<ul>\n<li>Broken Machines - The errors are very difficult to fix - Understanding how the flow of data works.</li>\n<li>Don't just copy, paste and run code - Understand it first.</li>\n<li>A good experimental design: what to test?</li>\n<li>Output - Plotting results</li>\n<li>Save all results </li>\n</ul>\n<h3>Workflow - Call in for reinforcements</h3>\n<p>Your workflow whether it be a very small one or a much bigger one must be smooth and smart from the start. If you follow the workflow described below and stick to it consistently the chances are great that your project will become more appealing, interesting, and well organized (in addition to more manageable and less prone to bugs).</p>\n<h4>Look for existing code first</h4>\n<ul>\n<li>Python: walk over the code and check for libraries</li>\n<li>Coding at its very center is sharing, and sharing of code is the key point that which Python and other programming languages were founded. When you can't code something on your own and you have no one else to turn to, just go to the Internet and search out real-world examples, applications, and implementations in the popular open-source libraries mentioned below. The engineering of the programs you want is already present. You just need to adapt it to your problem and use it. </li>\n</ul>\n<blockquote>\n  <p>Don't forget to check its license</p>\n</blockquote>\n<ul>\n<li><p>Don´t overcomplicate it - Work in a simple environment that can be upgraded at any time.</p></li>\n<li><p>If you are new to the programming world, consider getting the Python programming language as your primary language.</p>\n<ul>\n<li>Even if you have more experience with another language like Java, Unity, or PHP, mastering Python will simplify your lives in the future for it is both versatile and simple, taking almost little to no time to learn.</li></ul></li>\n<li><p>Care where to grow connections - Kaggle is a great place for this. </p></li>\n<li><p>Choose wisely</p>\n<ul>\n<li>Pick one library at first, and master it before moving further.</li>\n<li>Consider finding the answer and solution to your problem before anything else, review them to see examples for similar problems.</li></ul></li>\n<li><p>Build your own problem </p>\n<ul>\n<li>See with your data how it works</li>\n<li>Prepare your data &lt;=&gt; change the data and learn how to manipulate it.</li></ul></li>\n<li><p>Choose your tools wisely </p>\n<ul>\n<li>Colab, Jupyter Ide</li>\n<li>Visualizing your problem and working on your code</li></ul></li>\n</ul>\n<h4>Dive into a fascinating aspect of NN</h4>\n<ul>\n<li>Understand the graph and what is a neural network all about</li>\n<li>Before Training a model<ul>\n<li>Visualize your data</li>\n<li>Preprocessing in a separate function</li>\n<li>Feature engineering</li></ul></li>\n<li>Use all data you can<ul>\n<li>Visualize the whole data</li>\n<li>Data augmentation (different ways to augment: horizontal flip, or scaling up).</li></ul></li>\n<li>Get your own labels to train<ul>\n<li>Easy to collect: Scraping</li>\n<li>When scraping, check your data for what you collect</li></ul></li>\n<li>Prepare your model <ul>\n<li>Intuitively, or build from known papers</li></ul></li>\n</ul>\n<h4>Make decisions during training</h4>\n<ul>\n<li>Have a logging directory</li>\n<li>Early stopping, regularization, and early stopping<br>\n(it needs to be different to classify different things)</li>\n<li>The loss function</li>\n<li>Understand the semantics of these functions</li>\n</ul>\n<h4>Share the code base</h4>\n<ul>\n<li>Code with config in mind<ul>\n<li>Have a config dictionary in mind</li>\n<li>Have a loading function from the config</li></ul></li>\n<li>Have the directories ready (logging, model files, training files, …)</li>\n<li>Build multiple helper functions for accelerating coding.</li>\n</ul>\n<h4>Visualize your data</h4>\n<ul>\n<li>Plot the data and problem (visualization, image pro)</li>\n<li>Practice the more realistic data to the real world. (not contrived, test your program on real and realistic examples:  images, audio streams…)</li>\n</ul>\n<h4>Preprocessing and processing data</h4>\n<ul>\n<li>Preprocessing stage could be an augmenting stage.</li>\n<li>Train your model on raw data and then on processed data (and possible other data as a whole)</li>\n</ul>\n<h4>Know your network</h4>\n<ul>\n<li>Explain what each layer does</li>\n<li>Plot the graph (observe the inner workings of Deep Learning)</li>\n<li>Learn the semantics of layers of your Network</li>\n</ul>\n<h4>Debugging, Debugging Skills &amp; Debugging Hacks</h4>\n<ul>\n<li>Python comes with a built-in debugger to inspect all your custom code.</li>\n<li>Knowing what's inside the code leads you efficiently through all the debugging processes. Use the experiences, and ideas that you have to solve the problematic part of your code.</li>\n<li>Work in a controlled environment: Plotting, and checking your intermediate outputs if something is going wrong.<ul>\n<li>Check easily where the error is. (rule-of-thumb: simple problem, check top to bottom, complicated problem check bottom-up, )</li>\n<li>Always check the results of your previous experiments: what worked? what didn't? Ask your self \"Why? Why not\".</li></ul></li>\n</ul>\n<h4>Find inspiration easily</h4>\n<ul>\n<li>Understand existing code: Efficiently and effectively utilize the code you bought.</li>\n</ul>\n<h4>The logic behind the code</h4>\n<ul>\n<li>Create your own NN to see how it works.</li>\n<li>Batch Normalization (BN)</li>\n<li>Actual derivation and calculation of gradient descent.</li>\n<li>Regularization: L1 &amp; L2</li>\n</ul>\n<h4>Developing a complete project</h4>\n<ul>\n<li>Record all your parameter in training.<ul>\n<li>Create a mini log version.</li>\n<li>Evaluating the model in different versions.</li>\n<li>Export the version to CVS file so it can be translated into a JSON file, and edit the path to suit your requests.</li>\n<li>Importing data from CSV for better reading/comparison.</li></ul></li>\n<li>Block losses, metrics, notes (gradient descent, …) all functions in separate directories.<ul>\n<li>Easily edit and change whole plans not scripts.</li>\n<li>Any modification, debugging mode.</li>\n<li>Network layers and modules</li></ul></li>\n</ul>\n<h4>Practical Deploying and environments</h4>\n<ul>\n<li>Compute budgets for your projects<ul>\n<li>Project environments (what people stores, how women store their things)<ul>\n<li>Project: different projects require different tools/envs/</li>\n<li>Functionalities: What you know already and what libraries you need to.</li>\n<li>Able to define what you want and expect (how your code works, and how<br>\nthe system is supposed to be)</li>\n<li>How to obtain different compute units (GPU, resources …)</li></ul></li></ul></li>\n<li>Preparing your data - label, and format your dataset.</li>\n<li>Preview your model</li>\n<li>Build the model from config.</li>\n<li>Train the model: pickle the model, ready to test and use.</li>\n</ul>\n<h4>Tools of the trade</h4>\n<ul>\n<li>Learn the library coding convention and software engineering skills in the beginning.</li>\n<li>Debugging skills in Pycharm<ul>\n<li>Debug and track your problem with Python; jupyter notebook or similar libraries.</li>\n<li>Print, count object/memory/number in print(RUNCODE) or other libraries.</li></ul></li>\n<li>Plotting or drawing graphs with Matplotlib - Mplcursors</li>\n<li>Understand the whole algorithm &amp; structures.</li>\n<li>Train in Jupyter Notebook and then apply it on Pycharm.</li>\n</ul>\n<h4>Discipline - Keeping scripts and code clean and understandable</h4>\n<ul>\n<li><p>Each experiment should include unique, dedicated metadata about when, how, and why it was performed.</p>\n<ul>\n<li>Keeping the code, experiments, and their related output organized and shareable across computers.</li>\n<li>An attempt to make the code more deterministic:</li>\n<li>Execute a record and create a directory containing the data and the code of that experiment in addition to the required data of the state of the machine, dependencies, and versions.</li></ul></li>\n<li><p>Ensure you'll be able to discover and reproduce your experiments.</p>\n<ul>\n<li>Document the code, the version of all libraries, and the data of the experiment that should be recovered and what is expected to be obtained.</li>\n<li>Creating a directory tree in a meaningful way.</li>\n<li>Code reads like a book - Organize the subject before you start.</li></ul></li>\n<li><p>Prepare meta-data</p>\n<ul>\n<li>Put all the information related to the criteria in which you employed the data in the stored experiment.</li>\n<li>Prepare the information you will use to recover the experiment.</li>\n<li>Include libraries, archive types, and files important to the experiment.</li></ul></li>\n<li><p>Put the model or indicators in an easy-to-understand place.</p>\n<ul>\n<li>Understand the relationship between the data and the record.</li>\n<li>Keep it readable and understandable.</li>\n<li>Build the record to be well described.</li></ul></li>\n<li><p>Each experiment should be described with the purpose to recover each experiment.</p>\n<ul>\n<li>Put commas inside the file to record the position of the file.</li></ul></li>\n<li><p>Store directories and sub-directories of assets to ease the process of understanding and retrieving the records when needed.</p></li>\n</ul>\n<blockquote>\n  <p>With the abundance of machine learning frameworks and the development of auto-differentiation libraries, code deployment becomes a crucial next step to Get Ready To Start Learning Deep Learning. Let's first look at some \"CNN\"s concepts.</p>\n</blockquote>\n<h4>Coding</h4>\n<h4>Critical Tips on Code Architecture</h4>\n<blockquote>\n  <p>Effective code architecture for facilitating \"Deep Learning Plug and Play\" is a critical aspect of scientific research and utmost productivity.</p>\n</blockquote>\n<h5>Tools of the Trade - Code Architecture</h5>\n<ul>\n<li><p><strong>Follow a video tutorial or a full course</strong></p>\n<ul>\n<li>All of these resources are provided by TensorFlow and PyTorch, the most widely used libraries by practitioners in the academic and industrial look-alike machine learning fields. There is hardly any dissimilarity between modules of these libraries, so going with TensorFlow will do you no difference in practice. These are the resources to go with. They will teach you the ins and outs of how to train deep models and how to build effective ones.</li>\n<li>These resources will make you understand how your computer does work beyond the abstraction level of libraries, for a better understanding of Algorithms. </li></ul></li>\n<li><p><strong>Starting Deep Learning Training by training on low resolution and then go on to higher resolution</strong> </p>\n<ul>\n<li>This has nothing to do with the network itself, it will just save you time if your model won't work on the first run: Time-saving is A MUST when you start. </li></ul></li>\n<li><p><strong>Prepare your data in machine-readable formats</strong> (Numpy arrays, Pytorch tensors, …)</p>\n<ul>\n<li>Cut corners by opting for machine-readable &amp; writable formats, which are JIT-compiled codes of NumPy and so on.</li>\n<li>Understand formatting to the very deep end. It will be important and will save you time. </li></ul></li>\n</ul>\n<h5>Bits and Pieces 🤝</h5>\n<ul>\n<li><strong>Good architectures solve 80% of the software</strong> - <strong>Make sure the code runs smoothly before scaling up the number of samples.</strong> (scaling up the number of samples and starting with huge data sets is not as relevant as a well-designed project.)</li>\n</ul>\n<h4>A Short Concluding Note</h4>\n<ul>\n<li>Good ideas on multiple topics are important. But stick to one at a time when starting out.</li>\n<li>In addition, the ability to invent time, experiment, draw and sketch out something new is the real engine of scientific progress.</li>\n</ul>\n<p>Good Luck!</p>",
      "rawMarkdown": "# 💡 Tips for Learning Deep Learning Faster\n\n\n#### Welcome to the area you are interested in! 🎉 🎊\n\nBuilding a NN is the easy and hard part, but the understanding is what's difficult. When you want to learn something, you start, precisely, with the topic that you are interested in.\n\nFirst Tip: Work on a problem you **really** care about, this way you can sustain the motivation needed to progress.\n\n#### General Knowledge\n\n* Get ready to obtain broad knowledge.\n* Read as much as you can - understand the main concepts\n\n* Know how to code & factorize a problem to code.\n  - Factorize in smaller parts first\n  - Math skills - Important.\n\n* Start with Numpy, python 3, and basic editor/text-editor\n* Plot your data, and associate the concept to your data.\n\n* Choose your tools \n  - Pycharm\n  - Colab\n  - evn: conda, tensorflow, jupyter, ...\n\n#### Reading and coding good and CLEAN (simple + short + concise) code\n  - Know where all the parameters and objects are.\n  - Learn from masters; study their thoughts on what your code should include.\n  - If you don't understand the code, draw some outline/data flow charts.\n  - Master clean, simple, short, and precise code well before you start to bend over backward in Python with hacking tricks to excessively fancy code ahead of your basic computer science training (for creative, artificial intelligence applications)\n\n#### Know how to read a good paper (Paper Reading Strategies).\n- Comprehend the materials: What are the selling points.\n- Understand and figure out concepts the writer is saying.\n- Try to implement the paper. (copy it and modify)\n\n#### Study before start coding : \n  - Check for correctness during coding.\n  - Prerequisite about CS training.\n\n#### Start Testing and the details\n- Broken Machines - The errors are very difficult to fix - Understanding how the flow of data works.\n- Don't just copy, paste and run code - Understand it first.\n- A good experimental design: what to test?\n- Output - Plotting results\n- Save all results \n\n### Workflow - Call in for reinforcements\n\nYour workflow whether it be a very small one or a much bigger one must be smooth and smart from the start. If you follow the workflow described below and stick to it consistently the chances are great that your project will become more appealing, interesting, and well organized (in addition to more manageable and less prone to bugs).\n\n#### Look for existing code first\n- Python: walk over the code and check for libraries\n- Coding at its very center is sharing, and sharing of code is the key point that which Python and other programming languages were founded. When you can't code something on your own and you have no one else to turn to, just go to the Internet and search out real-world examples, applications, and implementations in the popular open-source libraries mentioned below. The engineering of the programs you want is already present. You just need to adapt it to your problem and use it. \n\n> Don't forget to check its license\n\n- Don´t overcomplicate it - Work in a simple environment that can be upgraded at any time.\n- If you are new to the programming world, consider getting the Python programming language as your primary language.\n  - Even if you have more experience with another language like Java, Unity, or PHP, mastering Python will simplify your lives in the future for it is both versatile and simple, taking almost little to no time to learn.\n\n- Care where to grow connections - Kaggle is a great place for this. \n\n- Choose wisely\n  - Pick one library at first, and master it before moving further.\n  - Consider finding the answer and solution to your problem before anything else, review them to see examples for similar problems.\n\n- Build your own problem \n  - See with your data how it works\n  - Prepare your data <=> change the data and learn how to manipulate it.\n\n- Choose your tools wisely \n  - Colab, Jupyter Ide\n  - Visualizing your problem and working on your code\n\n#### Dive into a fascinating aspect of NN \n- Understand the graph and what is a neural network all about\n- Before Training a model\n  - Visualize your data\n  - Preprocessing in a separate function\n  - Feature engineering\n- Use all data you can\n  - Visualize the whole data\n  - Data augmentation (different ways to augment: horizontal flip, or scaling up).\n- Get your own labels to train\n  - Easy to collect: Scraping\n  - When scraping, check your data for what you collect\n- Prepare your model \n  - Intuitively, or build from known papers\n\n#### Make decisions during training\n- Have a logging directory\n- Early stopping, regularization, and early stopping\n (it needs to be different to classify different things)\n- The loss function\n- Understand the semantics of these functions\n\n#### Share the code base \n- Code with config in mind\n  - Have a config dictionary in mind\n  - Have a loading function from the config\n- Have the directories ready (logging, model files, training files, ...)\n- Build multiple helper functions for accelerating coding.\n\n#### Visualize your data \n- Plot the data and problem (visualization, image pro)\n- Practice the more realistic data to the real world. (not contrived, test your program on real and realistic examples:  images, audio streams...)\n\n#### Preprocessing and processing data\n- Preprocessing stage could be an augmenting stage.\n- Train your model on raw data and then on processed data (and possible other data as a whole)\n\n#### Know your network \n- Explain what each layer does\n- Plot the graph (observe the inner workings of Deep Learning)\n- Learn the semantics of layers of your Network\n\n#### Debugging, Debugging Skills & Debugging Hacks\n  - Python comes with a built-in debugger to inspect all your custom code.\n  - Knowing what's inside the code leads you efficiently through all the debugging processes. Use the experiences, and ideas that you have to solve the problematic part of your code.\n  - Work in a controlled environment: Plotting, and checking your intermediate outputs if something is going wrong.\n- Check easily where the error is. (rule-of-thumb: simple problem, check top to bottom, complicated problem check bottom-up, )\n- Always check the results of your previous experiments: what worked? what didn't? Ask your self \"Why? Why not\".\n\n#### Find inspiration easily \n- Understand existing code: Efficiently and effectively utilize the code you bought.\n\n#### The logic behind the code \n- Create your own NN to see how it works.\n- Batch Normalization (BN)\n- Actual derivation and calculation of gradient descent.\n- Regularization: L1 & L2\n\n#### Developing a complete project\n- Record all your parameter in training.\n  - Create a mini log version.\n  - Evaluating the model in different versions.\n  - Export the version to CVS file so it can be translated into a JSON file, and edit the path to suit your requests.\n  - Importing data from CSV for better reading/comparison.\n- Block losses, metrics, notes (gradient descent, ...) all functions in separate directories.\n  - Easily edit and change whole plans not scripts.\n  - Any modification, debugging mode.\n  - Network layers and modules\n\n#### Practical Deploying and environments\n- Compute budgets for your projects\n  - Project environments (what people stores, how women store their things)\n      - Project: different projects require different tools/envs/\n      - Functionalities: What you know already and what libraries you need to.\n      - Able to define what you want and expect (how your code works, and how\n      the system is supposed to be)\n      - How to obtain different compute units (GPU, resources ...)\n- Preparing your data - label, and format your dataset.\n- Preview your model\n- Build the model from config.\n- Train the model: pickle the model, ready to test and use.\n\n#### Tools of the trade\n- Learn the library coding convention and software engineering skills in the beginning.\n- Debugging skills in Pycharm\n  - Debug and track your problem with Python; jupyter notebook or similar libraries.\n  - Print, count object/memory/number in print(RUNCODE) or other libraries.\n- Plotting or drawing graphs with Matplotlib - Mplcursors\n- Understand the whole algorithm & structures.\n- Train in Jupyter Notebook and then apply it on Pycharm.\n\n#### Discipline - Keeping scripts and code clean and understandable\n\n- Each experiment should include unique, dedicated metadata about when, how, and why it was performed.\n  - Keeping the code, experiments, and their related output organized and shareable across computers.\n  - An attempt to make the code more deterministic:\n  - Execute a record and create a directory containing the data and the code of that experiment in addition to the required data of the state of the machine, dependencies, and versions.\n\n- Ensure you'll be able to discover and reproduce your experiments.\n  - Document the code, the version of all libraries, and the data of the experiment that should be recovered and what is expected to be obtained.\n  - Creating a directory tree in a meaningful way.\n    - Code reads like a book - Organize the subject before you start.\n\n- Prepare meta-data\n  - Put all the information related to the criteria in which you employed the data in the stored experiment.\n  - Prepare the information you will use to recover the experiment.\n  - Include libraries, archive types, and files important to the experiment.\n\n- Put the model or indicators in an easy-to-understand place.\n  - Understand the relationship between the data and the record.\n  - Keep it readable and understandable.\n  - Build the record to be well described.\n\n- Each experiment should be described with the purpose to recover each experiment.\n  - Put commas inside the file to record the position of the file.\n- Store directories and sub-directories of assets to ease the process of understanding and retrieving the records when needed.\n\n> With the abundance of machine learning frameworks and the development of auto-differentiation libraries, code deployment becomes a crucial next step to Get Ready To Start Learning Deep Learning. Let's first look at some \"CNN\"s concepts.\n\n#### Coding\n\n#### Critical Tips on Code Architecture\n\n> Effective code architecture for facilitating \"Deep Learning Plug and Play\" is a critical aspect of scientific research and utmost productivity.\n\n##### Tools of the Trade - Code Architecture\n - **Follow a video tutorial or a full course**\n- All of these resources are provided by TensorFlow and PyTorch, the most widely used libraries by practitioners in the academic and industrial look-alike machine learning fields. There is hardly any dissimilarity between modules of these libraries, so going with TensorFlow will do you no difference in practice. These are the resources to go with. They will teach you the ins and outs of how to train deep models and how to build effective ones.\n- These resources will make you understand how your computer does work beyond the abstraction level of libraries, for a better understanding of Algorithms. \n \n \n - **Starting Deep Learning Training by training on low resolution and then go on to higher resolution** \n- This has nothing to do with the network itself, it will just save you time if your model won't work on the first run: Time-saving is A MUST when you start. \n\n - **Prepare your data in machine-readable formats** (Numpy arrays, Pytorch tensors, ...)\n- Cut corners by opting for machine-readable & writable formats, which are JIT-compiled codes of NumPy and so on.\n- Understand formatting to the very deep end. It will be important and will save you time. \n\n##### Bits and Pieces 🤝\n- **Good architectures solve 80% of the software** - **Make sure the code runs smoothly before scaling up the number of samples.** (scaling up the number of samples and starting with huge data sets is not as relevant as a well-designed project.)\n\n\n#### A Short Concluding Note \n- Good ideas on multiple topics are important. But stick to one at a time when starting out.\n- In addition, the ability to invent time, experiment, draw and sketch out something new is the real engine of scientific progress.\n\nGood Luck!",
      "votes": 1
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1817888": "# 💡 Tips for Learning Deep Learning Faster\n\n\n#### Welcome to the area you are interested in! 🎉 🎊\n\nBuilding a NN is the easy and hard part, but the understanding is what's difficult. When you want to learn something, you start, precisely, with the topic that you are interested in.\n\nFirst Tip: Work on a problem you **really** care about, this way you can sustain the motivation needed to progress.\n\n#### General Knowledge\n\n* Get ready to obtain broad knowledge.\n* Read as much as you can - understand the main concepts\n\n* Know how to code & factorize a problem to code.\n  - Factorize in smaller parts first\n  - Math skills - Important.\n\n* Start with Numpy, python 3, and basic editor/text-editor\n* Plot your data, and associate the concept to your data.\n\n* Choose your tools \n  - Pycharm\n  - Colab\n  - evn: conda, tensorflow, jupyter, ...\n\n#### Reading and coding good and CLEAN (simple + short + concise) code\n  - Know where all the parameters and objects are.\n  - Learn from masters; study their thoughts on what your code should include.\n  - If you don't understand the code, draw some outline/data flow charts.\n  - Master clean, simple, short, and precise code well before you start to bend over backward in Python with hacking tricks to excessively fancy code ahead of your basic computer science training (for creative, artificial intelligence applications)\n\n#### Know how to read a good paper (Paper Reading Strategies).\n- Comprehend the materials: What are the selling points.\n- Understand and figure out concepts the writer is saying.\n- Try to implement the paper. (copy it and modify)\n\n#### Study before start coding : \n  - Check for correctness during coding.\n  - Prerequisite about CS training.\n\n#### Start Testing and the details\n- Broken Machines - The errors are very difficult to fix - Understanding how the flow of data works.\n- Don't just copy, paste and run code - Understand it first.\n- A good experimental design: what to test?\n- Output - Plotting results\n- Save all results \n\n### Workflow - Call in for reinforcements\n\nYour workflow whether it be a very small one or a much bigger one must be smooth and smart from the start. If you follow the workflow described below and stick to it consistently the chances are great that your project will become more appealing, interesting, and well organized (in addition to more manageable and less prone to bugs).\n\n#### Look for existing code first\n- Python: walk over the code and check for libraries\n- Coding at its very center is sharing, and sharing of code is the key point that which Python and other programming languages were founded. When you can't code something on your own and you have no one else to turn to, just go to the Internet and search out real-world examples, applications, and implementations in the popular open-source libraries mentioned below. The engineering of the programs you want is already present. You just need to adapt it to your problem and use it. \n\n> Don't forget to check its license\n\n- Don´t overcomplicate it - Work in a simple environment that can be upgraded at any time.\n- If you are new to the programming world, consider getting the Python programming language as your primary language.\n  - Even if you have more experience with another language like Java, Unity, or PHP, mastering Python will simplify your lives in the future for it is both versatile and simple, taking almost little to no time to learn.\n\n- Care where to grow connections - Kaggle is a great place for this. \n\n- Choose wisely\n  - Pick one library at first, and master it before moving further.\n  - Consider finding the answer and solution to your problem before anything else, review them to see examples for similar problems.\n\n- Build your own problem \n  - See with your data how it works\n  - Prepare your data <=> change the data and learn how to manipulate it.\n\n- Choose your tools wisely \n  - Colab, Jupyter Ide\n  - Visualizing your problem and working on your code\n\n#### Dive into a fascinating aspect of NN \n- Understand the graph and what is a neural network all about\n- Before Training a model\n  - Visualize your data\n  - Preprocessing in a separate function\n  - Feature engineering\n- Use all data you can\n  - Visualize the whole data\n  - Data augmentation (different ways to augment: horizontal flip, or scaling up).\n- Get your own labels to train\n  - Easy to collect: Scraping\n  - When scraping, check your data for what you collect\n- Prepare your model \n  - Intuitively, or build from known papers\n\n#### Make decisions during training\n- Have a logging directory\n- Early stopping, regularization, and early stopping\n (it needs to be different to classify different things)\n- The loss function\n- Understand the semantics of these functions\n\n#### Share the code base \n- Code with config in mind\n  - Have a config dictionary in mind\n  - Have a loading function from the config\n- Have the directories ready (logging, model files, training files, ...)\n- Build multiple helper functions for accelerating coding.\n\n#### Visualize your data \n- Plot the data and problem (visualization, image pro)\n- Practice the more realistic data to the real world. (not contrived, test your program on real and realistic examples:  images, audio streams...)\n\n#### Preprocessing and processing data\n- Preprocessing stage could be an augmenting stage.\n- Train your model on raw data and then on processed data (and possible other data as a whole)\n\n#### Know your network \n- Explain what each layer does\n- Plot the graph (observe the inner workings of Deep Learning)\n- Learn the semantics of layers of your Network\n\n#### Debugging, Debugging Skills & Debugging Hacks\n  - Python comes with a built-in debugger to inspect all your custom code.\n  - Knowing what's inside the code leads you efficiently through all the debugging processes. Use the experiences, and ideas that you have to solve the problematic part of your code.\n  - Work in a controlled environment: Plotting, and checking your intermediate outputs if something is going wrong.\n- Check easily where the error is. (rule-of-thumb: simple problem, check top to bottom, complicated problem check bottom-up, )\n- Always check the results of your previous experiments: what worked? what didn't? Ask your self \"Why? Why not\".\n\n#### Find inspiration easily \n- Understand existing code: Efficiently and effectively utilize the code you bought.\n\n#### The logic behind the code \n- Create your own NN to see how it works.\n- Batch Normalization (BN)\n- Actual derivation and calculation of gradient descent.\n- Regularization: L1 & L2\n\n#### Developing a complete project\n- Record all your parameter in training.\n  - Create a mini log version.\n  - Evaluating the model in different versions.\n  - Export the version to CVS file so it can be translated into a JSON file, and edit the path to suit your requests.\n  - Importing data from CSV for better reading/comparison.\n- Block losses, metrics, notes (gradient descent, ...) all functions in separate directories.\n  - Easily edit and change whole plans not scripts.\n  - Any modification, debugging mode.\n  - Network layers and modules\n\n#### Practical Deploying and environments\n- Compute budgets for your projects\n  - Project environments (what people stores, how women store their things)\n      - Project: different projects require different tools/envs/\n      - Functionalities: What you know already and what libraries you need to.\n      - Able to define what you want and expect (how your code works, and how\n      the system is supposed to be)\n      - How to obtain different compute units (GPU, resources ...)\n- Preparing your data - label, and format your dataset.\n- Preview your model\n- Build the model from config.\n- Train the model: pickle the model, ready to test and use.\n\n#### Tools of the trade\n- Learn the library coding convention and software engineering skills in the beginning.\n- Debugging skills in Pycharm\n  - Debug and track your problem with Python; jupyter notebook or similar libraries.\n  - Print, count object/memory/number in print(RUNCODE) or other libraries.\n- Plotting or drawing graphs with Matplotlib - Mplcursors\n- Understand the whole algorithm & structures.\n- Train in Jupyter Notebook and then apply it on Pycharm.\n\n#### Discipline - Keeping scripts and code clean and understandable\n\n- Each experiment should include unique, dedicated metadata about when, how, and why it was performed.\n  - Keeping the code, experiments, and their related output organized and shareable across computers.\n  - An attempt to make the code more deterministic:\n  - Execute a record and create a directory containing the data and the code of that experiment in addition to the required data of the state of the machine, dependencies, and versions.\n\n- Ensure you'll be able to discover and reproduce your experiments.\n  - Document the code, the version of all libraries, and the data of the experiment that should be recovered and what is expected to be obtained.\n  - Creating a directory tree in a meaningful way.\n    - Code reads like a book - Organize the subject before you start.\n\n- Prepare meta-data\n  - Put all the information related to the criteria in which you employed the data in the stored experiment.\n  - Prepare the information you will use to recover the experiment.\n  - Include libraries, archive types, and files important to the experiment.\n\n- Put the model or indicators in an easy-to-understand place.\n  - Understand the relationship between the data and the record.\n  - Keep it readable and understandable.\n  - Build the record to be well described.\n\n- Each experiment should be described with the purpose to recover each experiment.\n  - Put commas inside the file to record the position of the file.\n- Store directories and sub-directories of assets to ease the process of understanding and retrieving the records when needed.\n\n> With the abundance of machine learning frameworks and the development of auto-differentiation libraries, code deployment becomes a crucial next step to Get Ready To Start Learning Deep Learning. Let's first look at some \"CNN\"s concepts.\n\n#### Coding\n\n#### Critical Tips on Code Architecture\n\n> Effective code architecture for facilitating \"Deep Learning Plug and Play\" is a critical aspect of scientific research and utmost productivity.\n\n##### Tools of the Trade - Code Architecture\n - **Follow a video tutorial or a full course**\n- All of these resources are provided by TensorFlow and PyTorch, the most widely used libraries by practitioners in the academic and industrial look-alike machine learning fields. There is hardly any dissimilarity between modules of these libraries, so going with TensorFlow will do you no difference in practice. These are the resources to go with. They will teach you the ins and outs of how to train deep models and how to build effective ones.\n- These resources will make you understand how your computer does work beyond the abstraction level of libraries, for a better understanding of Algorithms. \n \n \n - **Starting Deep Learning Training by training on low resolution and then go on to higher resolution** \n- This has nothing to do with the network itself, it will just save you time if your model won't work on the first run: Time-saving is A MUST when you start. \n\n - **Prepare your data in machine-readable formats** (Numpy arrays, Pytorch tensors, ...)\n- Cut corners by opting for machine-readable & writable formats, which are JIT-compiled codes of NumPy and so on.\n- Understand formatting to the very deep end. It will be important and will save you time. \n\n##### Bits and Pieces 🤝\n- **Good architectures solve 80% of the software** - **Make sure the code runs smoothly before scaling up the number of samples.** (scaling up the number of samples and starting with huge data sets is not as relevant as a well-designed project.)\n\n\n#### A Short Concluding Note \n- Good ideas on multiple topics are important. But stick to one at a time when starting out.\n- In addition, the ability to invent time, experiment, draw and sketch out something new is the real engine of scientific progress.\n\nGood Luck!"
  }
}