{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Dive in Computer Vision","metadata":{}},{"cell_type":"markdown","source":"\n
\n \nBackground\n \nIntroduction\n \nHow Computer Store Images\n \nWhat really computer vision is?\n \nLet's Explore Tasks\n \nTraditional Approach vs Deep Learning Approach for CV\n\nRecent developments\n \nImplementation of Convolutional Neural Networks\n \nSystem Requirements\n \nAlgorithms\n \nCloud Platforms\n \nCompete and Learn on Kaggle\n \nRecent Trends and Future Scope\n \nConclusion\n \nReferences and Citations\n \n
","metadata":{}},{"cell_type":"markdown","source":"

Background

\n\n
\n

\nComputer Vision is one of the most promising feild and evolutionary technology. It is base of multiple technologies like Augmented Reality, Virtual Reality, Metaverse, Robotics and many more. Everyone gets interacted with computer vision, but how? You use Snapchat filters, Instagram filters all these use computer vision under the hood. This is the reason why their comes hat exactly on your head or dog's ears gets replaced with your ears in photo. Now to make things more interesting, Tesla (driverless car), biometric authentication, image/video generation from text all these have computer vision implemented in it.\n

\n \n

\nIn this essay my goal will be to introduce to you to one of the fast advancing technology in simple and beginner friendly manner. After reading this you will get familier with computer vision. I will more talk on images in this essay but as a video is collection of sequence of images i.e frames so it will be implemented on that also.\n

\n

\nI will talk about pre-trained models, transfer learning, cloud platforms for CV, CV tasks, use cases, recent trends and many more.\n

\n \n

\nI will be talking about recent advancements and past advancements also to make you more aquainted with CV. Deep Learning revolutionarized computer vision, so we will focus more on deep learning approach on computer vision rather than traditional approach. We will compare both of them somehow. Convolutional Neural Networks (CNN) is base of lots of deep learning CV algorithms s it is important for us to now about CNN so then we can comfortably understand algorithms like ResNet, YOLO V8 etc. Then I will show you how we can use Kaggle platform so we can learn more about computer vision.\n

\n \n

\nSo without a delay, let's dive in computer vision and be ready to explore the promising technology of the future.\n

\n \n
","metadata":{}},{"cell_type":"markdown","source":"

Introduction

\n\n
\n

\nIn the world of living beings, our eyes serve as extraordinary sensory organs, allowing us to observe and perceive our surroundings within a defined range. Similarly, computers are equipped with Graphics Processing Units (GPUs) that enable them to process and render images and videos on screens or monitors. When we talk about videos, we refer to a series of images, known as frames, that are captured and displayed at specific intervals [1].

\n\n

\nFor computers, images are interpreted as numerical values called pixels, which are stored in matrices. These images can be grayscale, representing black and white, or in RGB format, capturing colors using red, green, and blue channels.

\n\n

\nComputer vision (CV), a field of artificial intelligence (AI), empowers computers and systems to extract meaningful information from digital images, videos, and other visual inputs. This extracted information can then be utilized to perform intended actions based on the analyzed data [2]. Computer vision finds applications in diverse domains such as self-driving cars (e.g., Tesla), augmented reality apps (e.g., Snapchat filters, Pokémon Go), healthcare, robotics, manufacturing, automotive, metaverse, extended reality (XR), and more [3].

\n\n

\nNumerous algorithms are available for computer vision, including Convolutional Neural Networks (CNN), ResNet, You Only Look Once (YOLO), and many others [4]. CNNs form the foundation of many popular and widely adopted algorithms today.

\n\n

\nSelecting the appropriate algorithm depends on several factors, including the specific CV application being worked on (e.g., image recognition) and the characteristics of the data involved, such as dataset size and complexity.

\n\n

\nMany advanced algorithms, such as YOLO, ResNet, AlexNet, InceptionNet, and SSD (Single Shot MultiBox Detector), are built upon the CNN architecture [5][6]. Thus, understanding CNNs in depth is crucial for effectively working with these algorithms.

\n\n

\nImplementing computer vision requires utilizing libraries, frameworks, and techniques. Transfer learning is one such technique commonly used. Cloud platforms like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) offer a range of services and tools to facilitate the implementation of computer vision tasks. Successful execution of computer vision projects also relies on robust hardware and software support. Platforms like Kaggle host numerous competitions and provide resources for advancing computer vision capabilities.

\n \n

\nHere computer is broader term we are refering to the machines, it can be robots, mobile phones, desktops etc. There is a vast amount of knowledge to explore in computer vision. Let's delve into it, and I assure you that you will find it fascinating.

\n\n
","metadata":{"execution":{"iopub.status.busy":"2023-07-04T18:51:00.285869Z","iopub.execute_input":"2023-07-04T18:51:00.286182Z","iopub.status.idle":"2023-07-04T18:51:00.322151Z","shell.execute_reply.started":"2023-07-04T18:51:00.286158Z","shell.execute_reply":"2023-07-04T18:51:00.320693Z"}}},{"cell_type":"markdown","source":"

How computers store images

\n\n
\n

\nDo computers store images in their storage memory as human store images in the brain? Answer is a big No.

\n
\n

Humans store images in their memory through visual perception and cognitive processes.\nWhen we see an image, our eyes capture the light reflecting off objects, and the information is transmitted to our brain through the optic nerves.\nThe brain processes and interprets the visual information, storing it in our memory.\nHuman memory is not a precise representation of the image but rather a combination of visual details, emotions, and associations connected to that image.\n The memory of an image in our brain can fade or be altered over time[7].

\n\n

\nUnlike human, computer store images in the form of numbers called as pixels in a Matrix. Their are mainly two types of channels, Grayscale (Black and White) and RGB(Red Green Blue).

\n \n

\nComputers store grayscale images and RGB images in storage memory as a matrix of numbers. The matrix has dimensions that match the dimensions of the image, and each element in the matrix represents the brightness or color of a pixel in the image.

\n\n

\nFor example, a grayscale image with dimensions 100x100 would be stored as a matrix with dimensions 100x100. Each element in the matrix would be a number between 0 and 255, representing the brightness of the corresponding pixel in the image.

\n\n

\nAn RGB image with dimensions 100x100 would be stored as a matrix with dimensions 100x100x3. Each element in the first dimension of the matrix would represent a pixel in the image. The next three elements in the matrix would represent the red, green, and blue channels of the pixel, respectively.

\n\n

Grayscale

\n

\nThese images have one color channel, which represents the whole image. The intensity of each pixel in a grayscale image is represented by a number between 0 and 255, where 0 represents black and 255 represents white.[8]

\n \n \n
\n
\n\n

example of number 8 in Grayscale [8]

\n\n\n

RGB Image

\n\n

\nThese images have three color channel, which represents the whole image. The intensity of each pixel in a RGB image is represented by a number between 0 and 255, where 0 represents the absence of that color channel and 255 represents the maximum intensity of that color channel.

\n\n

\nThe red channel represents the intensity of red light in each pixel.\nThe green channel represents the intensity of green light in each pixel.\nThe blue channel represents the intensity of blue light in each pixel.

\n\n

\nThe combination of these three color channels at different intensities produces the full range of colors in an RGB image. For example, an RGB pixel with values (0, 0, 0) represents black (no contribution from any color channel), and a pixel with values (255, 255, 255) represents white (maximum intensity in all color channels).

\n\n\n\n\n

Example - Mario RGB Image [9]

\n\n\n

File Formats

\n\n

\nImages are stored in specific file formats that determine how the pixel data is organized, encoded, and compressed

\n\n

\nExamples -

\n\n

\nJPEG: JPEG is a lossy compression format, which means that some data is lost when the image is compressed. This makes JPEG files smaller, but it also means that the image quality may be reduced.

\n\n

\nPNG: PNG is a lossless compression format, which means that no data is lost when the image is compressed. This makes PNG files larger, but it also means that the image quality is preserved.

\n\n

\nGIF: GIF is a format that is used for animated images. GIF files can contain multiple frames, which are displayed one after the other to create the illusion of movement.

\n\n

\nBMP: BMP is a format that is used for uncompressed images. BMP files are large, but they provide the best image quality.

\n\n

\nOther images file formats are TIFF, EPS, RAW Image Files (.raw, .cr2, .nef, .orf, .sr2, and more) [10].

\n\n
\n\n","metadata":{}},{"cell_type":"markdown","source":"

What really Computer Vision is?

\n\n
\n

\nComputer vision is a field of artificial intelligence that enables computers and systems to derive meaningful information from digital images, videos and other visual inputs and do intended tasks based on that information. If AI enables computers to think, computer vision enables them to see, observe and understand.[2]

\n \n

\nThe goal of computer vision is to extract meaningful information from visual data, such as images or videos, and make intelligent decisions or perform specific tasks based on that information. This includes tasks such as image classification, object detection and recognition, image segmentation, object tracking, etc.[11]

\n \n

\nComputer vision algorithms and models employ various techniques to process visual data, including image preprocessing, feature extraction, pattern recognition, and machine learning. Deep learning has played a significant role in advancing computer vision by enabling algorithms to automatically learn and recognize complex patterns and structures within images.[11]

\n \n

\nToday, we use deep learning for computer vision, and all of the work is done by neural networks. When we provide neural networks with lots and lots of labeled images of a particular object, they are able to identify patterns in them.

\n \n

\nAs video is flow of images in sequence(frames), what we do on images get applied to videos.

\n\n\n\n
","metadata":{}},{"cell_type":"markdown","source":"

Let's Explore Tasks

\n\n
\n \nComputer Vision Tasks mainly fall under four sub-categories i.e. Classification, Detection, Recognition, Segmentation. All these tasks seems to be related and overlapped with each other having some differences. Some of the most common tasks are as follows :\n \n

Image Classification :

\n

\nIt deals with processing the entire image as a whole and assigning a specific label to it. This is the task of assigning an image to a category based on its content. For example, an image classification algorithm might be able to identify an image as a cat, a dog, or a monkey. When we go one step forward and locate the object in image then it is termed as image classification with localization. Their are mainly three types of classification :

\n

\nBinary image classification : Distinguish between two classes of images. For example, distinguish between cat images and dog images.

\n \n

\nMulticlass image classification : Distinguish between multiple classes of images. For example, distinguish between cat images, dog images, bird images, and car images.

\n \n

\nMultilabel image classification : Assign multiple labels to an image. For example, it might be trained to assign the labels \"cat\", \"dog\", and \"animal\" to an image of a cat. \n

\n \n\n

Image of single label classificatin Vs multi-label classification [source : Google Images]

\n \n

Object Recognition and Image Recognition :

\n\n

\nThese terms are often confused and used interchangebly. Object recognition is the task of identifying patterns within an image. For example, an object recognition algorithm might be able to identify a cat in an image. Image recognition is the task of assigning an image to a category based on its content. For example, an image recognition algorithm might be able to identify an image as a cat, a dog, or a car.\n

\n \n

Object Detection

\n

\nObject detection is the task of finding and locating objects in an image or video. This is done by identifying class-specific features in the image or video and then drawing bounding boxes around the objects. The classes of objects that can be detected can be anything that the detection model has been trained on, such as cars, animals, humans, or even specific objects like traffic signs or stoplights.\n

\n\n\n \n

In above image trees, human, aieroplane are detected [30]

\n \n

Instance Segmentation

\n

\n Instance Segmentation is a computer vision task that involves identifying and separating individual objects within an image, including detecting the boundaries of each object and assigning a unique label to each object. The goal of instance segmentation is to produce a pixel-wise segmentation map of the image, where each pixel is assigned to a specific object instance.\n

\n \n

Semantic Segmentation

\n

\nSemantic segmentation is a computer vision task that assigns a semantic label to each pixel in an image. This is done by identifying the objects in the image and then assigning a label to each pixel based on the object that it belongs to. The goal is to produce a dense pixel-wise segmentation map of an image, where each pixel is assigned to a specific class or object.\n\nSome example benchmarks for this task are Cityscapes, PASCAL VOC, and ADE20K. These benchmarks provide datasets of images with ground-truth labels, which can be used to train and evaluate semantic segmentation algorithms. Models are usually evaluated with the Mean Intersection-Over-Union (Mean IoU) and Pixel Accuracy metrics.\n

\n \n\n

Instance and Semantic Segmentation [source: Analytics Vidya]

\n \n \n

Image Captioning

\n

It is looking at an image and describing what is happening in the image. The image given below contains annotations or labels which describe what is happening in the Image which should give you a good idea about what Image Captioning does.

\n\n

Captioned Image [29]

\n \n

Face Detection

\n

Face detection is a problem which computer vision attempts to solve locating and recognizing one or more faces in a picture. Locating faces in a picture is based on finding the exact coordinates of a face in a picture, while the localization is in relation to delimiting the range of a picture through a bounding box (a bordered frame) around a face.\n\n

\n\n

Face Detection[31]

\n
\n","metadata":{}},{"cell_type":"markdown","source":"

Traditional Approach vs Deep Learning Approach for CV

\n\n\n
\n \n

\nIn the traditional approach computer vision used to primarily rely on algorithms and techniques for image processing. The central task of computer vision involved extracting various characteristics from images, such as color, edges, corners, and objects. Detecting these features constituted the initial step in performing any computer vision-related task.

\n\n

\nThese features were manually engineered by humans, and the accuracy and reliability of the models depended heavily on the effectiveness of the feature extraction methods employed. In the realm of traditional computer vision, algorithms like SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), and BRIEF (Binary Robust Independent Elementary Features) played a pivotal role in extracting features from raw images.

\n \n

\nHowever, a challenge with this feature extraction approach in image classification was the need to select which features to focus on in each specific image. As the number of classification classes increased or the image quality degraded, it became exceedingly difficult to handle these scenarios using traditional computer vision algorithms. [20]

\n \n
\n \n \n [20]\n\n \n
\n \n\n>

If you want to teach a [deep] neural network to recognize a cat, for instance, you don’t tell it to look for whiskers, ears, fur, and eyes. You simply show it thousands and thousands of photos of cats, and eventually it works things out. If it keeps misclassifying foxes as cats, you don’t rewrite the code. You just keep coaching it. [20]

\n \n

The explosion began with the advent of convolutional neural networks (ConvNets) and their modified architectures. Today, it is often claimed that certain ConvNet models approach near-perfect accuracy in image classification challenges, occasionally outperforming the human eye!

\n\n

\nThe fundamental difference in the deep learning approach to computer vision lies in the concept of end-to-end learning. There is no longer a need to explicitly define features or engage in feature engineering. The neural network takes on this responsibility, simplifying the process significantly. [20]\n

\n \n
\n\n
","metadata":{}},{"cell_type":"markdown","source":"

Recent developments

\n\n
\n

Let tell a look at recent notable development in computer vision feild.

\n

Pre-trained Models and Transfer Learning :

\n

\nPre-trained models are machine learning models that have been trained on a large dataset of data. This data can be anything from images to text to code. Once a pre-trained model has been trained, it can be used to perform a variety of tasks.\n

\n

\nTransfer learning is a technique in machine learning where a model that has been trained on one task is used to improve the performance of a model that is being trained on a different task.\n

\n

\nPre-trained models work by learning the basic features of images and text through a process called feature extraction. Feature extraction is a process where the model learns to identify the most important features in an image or text. Once the model has learned the basic features, it can be used to perform a variety of tasks, such as image classification and object detection.\n

\n

\nPre-trained models are also significant because they can be used to transfer knowledge from one task to another. This means that a pre-trained model that has been trained on a large dataset of images can be used to improve the performance of a deep learning model that is being trained on a smaller dataset of images. Our accuracy gets improved and time, efforts are saved.\n

\n \n

Imagen - Diffusion Model :

\n

\nImagen is a text-to-image diffusion model that is able to generate photorealistic images from text descriptions. It is powered by a large transformer language model that understands text and a diffusion model that generates high-fidelity images.\n

\n

\nOne of the key discoveries of the Imagen team is that generic large language models (e.g. T5) that are pretrained on text-only corpora can be surprisingly effective at encoding text for image synthesis. This means that increasing the size of the language model in Imagen has a greater impact on the quality of the generated images than increasing the size of the image diffusion model.\n

\n

\nImagen achieves a new state-of-the-art FID score of 7.27 on the COCO dataset, without ever training on COCO. This means that Imagen is able to generate images that are as realistic as images in the COCO dataset, even though it was not trained on that dataset.

\n

\nTo assess text-to-image models more effectively, the Imagen team introduced DrawBench, a comprehensive and challenging benchmark for text-to-image models. Using DrawBench, the Imagen team compared Imagen to other recent methods, including VQ-GAN+CLIP, Latent Diffusion Models, and DALL-E 2. The results showed that human raters preferred Imagen over other models in side-by-side comparisons, both in terms of sample quality and image-text alignment. [34]

\n\n

Visualization of Imagen [33]\n

\n\n\n \n
\n","metadata":{}},{"cell_type":"markdown","source":"

Algorithms

\n\n
\n\n \n\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n\n\n\n\n \n
Traditional AlgorithmsDeep Learning Algorithms
SIFT (Scale-Invariant Feature Transform)\nConvolutional Neural Networks (CNN)\n
SURF (Speeded-Up Robust Features)AlexNet
BRIEF (Binary Robust Independent Elementary Features)\nVGG (Visual Geometry Group) Networks\n
HOG (Histogram of Oriented Gradients)\nGoogLeNet/Inception
Haar Cascade Classifiers\nResNet (Residual Network)\n
Canny Edge Detection\nDenseNet
Watershed Segmentation\nMobileNet
Harris Corner Detection\nYOLO (You Only Look Once)\n
Mean-Shift Tracking\nFaster R-CNN (Region-based Convolutional Neural Networks)\n
Lucas-Kanade Optical Flow\n\nMask R-CNN\n\n
\n\n\n

10 algorithms Traditional and Deep Learning [21]

\n\n
\n\n\n","metadata":{}},{"cell_type":"markdown","source":"

Implementation of Convolutional Neural Networks

\n\n
\n

Convolutional neural networks (CNNs) are a type of deep learning algorithm that can identify objects in images. They do this by learning to assign importance to different aspects of an image, such as edges, shapes, and colors. CNNs require less pre-processing than other classification algorithms, and they can learn to identify features without being explicitly programmed to do so.

\n\n\n

The architecture of a CNN is inspired by the way that neurons in the human brain are connected. Neurons in the visual cortex respond to stimuli only in a restricted region of the visual field, known as the receptive field. A collection of such fields overlap to cover the entire visual area. CNNs work in a similar way, by using filters to scan an image and identify features in different parts of the image.

\n \n
\n \n \n\n \n

CNN Algorithm implementation visuals [23]\n

\n\n
\n \n \n

\nCNN typically have five layers in it. They are convolutional layer, pooling layer, flatten layer, fully connected layer. Their is activation function which are widely used in CNN. \n

\n\n

Let's see all these :

\n \n \n

Input [ Image or frame (In case of video) ] :

\n

\nThis is ingest by CNN algorithm so it can perform all the computions, processing like feature engineering, feature scaling, feature extraction etc.\n

\n \n
\n \n

Covolutional Layer :

\n

\nIn mathematics (in particular, functional analysis), convolution is a mathematical operation on two functions (f and g) that produces a third function ( ) that expresses how the shape of one is modified by the other. The term convolution refers to both the result function and to the process of computing it.[25]

\n\n

\nA convolutional layer is a type of layer in a convolutional neural network (CNN) that extracts features from images. It does this by sliding a filter over the image and calculating the dot product of the filter and the image at each location. The filter is a small matrix of weights that is used to identify specific features in the image.

\n\n

\nThe output of the convolutional layer is a feature map, which is a representation of the image that highlights the features that were extracted by the filter. The feature maps are then passed to the next layer in the CNN, which may be another convolutional layer, a pooling layer, or a fully connected layer.

\n\n

\nConvolutional layers are the core of CNNs, and they are responsible for extracting the features that are used to classify or detect objects in images. The size and shape of the filters, as well as the number of convolutional layers in a CNN, are important factors that affect the performance of the CNN.

\n\n \n\n \n

Convolution operation on a MxNx3 image matrix with a 3x3x3 Kernel [22]\n

\n
\n \n

Pooling Layer\n

\n \n

\nA pooling layer is a type of layer in a convolutional neural network (CNN) that reduces the spatial dimensions of the feature maps while preserving the most important features. This helps to reduce the computational complexity of the CNN and to prevent overfitting.

\n \n\n

\nThere are two main types of pooling layers: max pooling and average pooling. Max pooling takes the maximum value from each region of the feature map, while average pooling takes the average value from each region of the feature map.

\n \n

\nThe size of the pooling window and the stride are important hyperparameters that affect the performance of the pooling layer. The pooling window is the size of the region that is pooled, and the stride is the number of pixels that are skipped between each region.

\n\n

\nPooling layers are typically used after convolutional layers in CNNs. This is because the convolutional layers extract features from the images, and the pooling layers reduce the spatial dimensions of the feature maps while preserving the most important features.\n

\n \n\n \n

Max Pooling Example [23]\n

\n \n
\n \n

Activation Function

\n

\nA neural network without an activation function is essentially just a linear regression model. The activation function does the non-linear transformation to the input making it capable to learn and perform more complex tasks.[26]

\n \n

\nThe most common activation function is the ReLU function. The ReLU function is a rectified linear unit function that outputs the input value if it is positive, and 0 if it is negative. The ReLU function is a very popular activation function because it is computationally efficient and can help to prevent overfitting.

\n\n

\nOther popular activation functions include the sigmoid function, the tanh function, and the leaky ReLU function. The sigmoid function outputs a value between 0 and 1, and is often used in classification problems. The tanh function outputs a value between -1 and 1, and is often used in regression problems. The leaky ReLU function is a variation of the ReLU function that outputs a small positive value if the input value is negative. This helps to prevent the vanishing gradient problem, which can occur with the ReLU function.

\n\n

\nThe choice of activation function depends on the specific task that the neural network is being used for. For example, the sigmoid function is often used in classification problems, while the ReLU function is often used in regression problems.

\n \n \n\n \n
\n \n

All major activation functions at a glance [27]

\n \n

Flattening Layer

\n

\nA flatten layer is a type of layer in a convolutional neural network (CNN) that converts the output of the convolutional layers into a one-dimensional vector. This is necessary because the fully connected layers that come after the convolutional layers can only process one-dimensional vectors.

\n\n

\nThe flatten layer does not have any weights or biases. It simply takes the output of the convolutional layers and reshapes it into a one-dimensional vector. This is a very simple layer, but it is essential for the proper functioning of CNNs.

\n \n

\nThe flatten layer is a common layer in CNNs. It is used in most CNN architectures that have convolutional layers and fully connected layers.\n

\n \n\n \n

Flattening of a 3x3 image matrix into a 9x1 vector [22]

\n\n
\n \n

Fully Connected Layer Layer

\n \n

\nA fully connected layer (FC layer) is a type of layer in a neural network that connects all neurons in one layer to all neurons in the next layer. FC layers are typically used at the end of a neural network, as they allow the network to make predictions or classifications.

\n \n

\nFC layers are similar to the layers in a traditional neural network. However, FC layers are typically used in CNNs after the convolutional layers have extracted features from the input image. This is because FC layers can learn more complex relationships between the features extracted by the convolutional layers.\n

\n \n
\n \n
","metadata":{}},{"cell_type":"markdown","source":"

System Requirements

\n\n
\n \n

\nWhen processing and analyzing the visual data like images and videos for extracting the patterns and information require strong hardware support as it works on huge amount of data which also can be big in size. Generally images and videos files have big sizes. So it is necessary to understand minimum configuration and recommended configuration so we can implement our computer vision tasks.

\n \n \n

\nIf we do not have minimum hardware support then components like CPU's and GPU's will get high load on it and eventually it will stop working. We can explore other option to work on computer vision tasks such as using cloud services provided by cloud providers like AWS provides AWS Rekognition, transfer learning, running notebooks on cloud like Google Colab etc. We will see these ways later on in this essay.

\n \n

\nCentral Processing Unit(CPU)

\n \n

\nA powerful CPU is essential for computer vision tasks. Multi-core processors, such as Intel Core i7 or AMD Ryzen series, are recommended as they can handle parallel processing required by many computer vision algorithms.

\n\n

\nGraphics Processing Unit(GPU)

\n \n

\nGPUs are highly beneficial for accelerating computer vision tasks, especially for deep learning-based approaches. GPUs with CUDA cores, such as NVIDIA GeForce or Quadro series, are commonly used to speed up training and inference of deep neural networks.

\n\n

\nRandom Access Memory(RAM)

\n \n

\nSufficient RAM is crucial for storing and processing large image datasets. The amount of RAM required depends on the size of the dataset and the complexity of the algorithms. At least 16GB or more RAM is generally recommended.

\n\n

\nStorage

\n \n

\nComputer vision tasks involve working with large amounts of image or video data, so having fast and ample storage is important. Solid State Drives (SSDs) are preferred over traditional Hard Disk Drives (HDDs) due to their faster read/write speeds.

\n\n

\nDedicated hardware

\n \n

\nFor specialized computer vision tasks or real-time applications, dedicated hardware accelerators like Field-Programmable Gate Arrays (FPGAs) or Tensor Processing Units (TPUs) can offer significant performance improvements. [13][14]

\n \n
","metadata":{}},{"cell_type":"markdown","source":"

Cloud Platforms

\n\n
\n

We can leverage cloud platforms in the development of computer vision applications. \nThe main benefits of developing computer vision apps on cloud are :

\n \n

\nSystem support : In CV we genearlly deals with huge amount of data, so high end processing/computational power is needed to run CV apps successsfully. Cloud platforms offer access to powerful computing resources, including GPUs (Graphics Processing Units) and TPUs (Tensor Processing Units), which are essential for training and running deep learning models used in computer vision tasks. Leveraging these resources in the cloud can significantly speed up the processing and analysis of visual data. \nEx. Kaggle Notebooks, Google Colab etc.

\n \n

\nPre-built APIs : Cloud providers offer pre-built APIs specifically designed for computer vision tasks. These APIs provide high-level functionalities that allow you to integrate computer vision capabilities into your applications without having to build everything from scratch.\nEx. AWS Rekognition, GCP Cloud Vision API, Azure Cognitive API

\n \n

\nStorage support : Dataset containing images and videos can be huge in size so it can cause storage problems if you have less storage capabilities in local machine. Cloud platform provides storage infrastructure to store data.

\n \n\n\n

Some cloud services provided by cloud providers on computer vision :

\n \n

\nGoogle Cloud Vision API: The Google Cloud Vision API is a powerful Computer Vision API that offers a wide range of features, including image and video recognition, OCR, facial detection, and object tracking. It is known for its accuracy and reliability, and it offers a simple and user-friendly API.

\n \n

\nAmazon Rekognition: Amazon Rekognition is another popular Computer Vision API that offers features such as object detection, facial recognition, text recognition, and celebrity recognition. It is known for its ease of use and scalability, and it integrates well with other Amazon Web Services.

\n \n

\nMicrosoft Azure Computer Vision API: The Microsoft Azure Computer Vision API offers features such as image and video recognition, OCR, and facial recognition. It is known for its accuracy and speed, and it offers a wide range of language support.

\n \n

\nOther services are Visual Recognition by IBM Watson, Clarifai etc. [16]

\n\n\n
\n\n\n","metadata":{}},{"cell_type":"markdown","source":"

Compete and Learn on Kaggle

\n\n
\n

\nComputer Vision is sub-field of data science. Kaggle is best place if you want to learn about data science. It has wide range of datasets available for data science including computer vision. Kaggle also hosts various competitions in which we can participate and learn. Some competitions also have huge price money. It have a big community of data science enthusiasts. We can use Kaggle Notebooks to code. We get community support if we get engage with Kaggle.

\n \n

\nWe get knowledge and information about latest developments in data science(includes computer vision) on Kaggle. It give us huge benefit to stay up-to-date in fast evolving field. Their are lots of discussions on Kaggle community and you get to learn from each other.

\n \n
\n \n

Some recent computer vision related competitons are -

\n \n

Google - American Sign Language Fingerspelling Recognition

\n \n

HuBMAP - Hacking the Human Vasculature

\n \n

CAFA 5 Protein Function Prediction

\n\n \n

Take a look for previous Kaggle competitions on computer vision by clicking here

\n \n [15]\n\n
","metadata":{}},{"cell_type":"markdown","source":"

Recent Trends and Future Scope

\n\n
\n\n

\nAutonomous vehicles : Computer vision algorithms play a crucial role in enabling self-driving vehicles by identifying objects and ensuring safe navigation. Techniques like auto-labelling are being adopted to advance autonomous driving technology.

\n

\nImage Generation : Image generation AI is a type of artificial intelligence (AI) that can create images from text descriptions. This is done by using a deep learning model that has been trained on a large dataset of images and text descriptions.The deep learning model learns to associate the text descriptions with the corresponding images. This allows the model to generate new images that match the text descriptions. Example, DALL-E 2

\n

\nIncreased use of edge computing : Real-time processing of visual data is driving the trend towards edge computing. Performing computations closer to the data source allows for quick and accurate decision-making without relying heavily on centralized servers or cloud-based systems.

\n \n

\nRobotics : Computer vision empowers robots to navigate, manipulate objects, and understand their environment. Robots can analyze images and video to detect objects, recognize human behavior, and perform tasks like grasping objects and navigating complex environments. Applications include customer service, education, and healthcare.

\n \n \n

\nHealthcare : Computer vision is utilized in medical image analysis to detect abnormalities and diseases in X-rays, CT scans, and MRIs. It assists with diagnosis, treatment planning, patient health monitoring, and robotic surgery. It also contributes to safety and security through surveillance and threat detection systems.

\n \n

\nRetail : Computer vision enables inventory management, automatic stock detection, and recognition of popular items in retail stores. It can be utilized for creating virtual fitting rooms and mirrors, allowing customers to try out items virtually.

\n \n

\n3D reconstruction : Techniques like Neural Radiance Fields (NeRF) have enabled the reconstruction of 2D images into 3D meshes. This has applications in recreating scenes, building models for virtual and augmented reality experiences, and enhancing interaction with digital environments.

\n \n

\nSpaceTech : Computer vision is used in space industry applications such as object detection in the sky using smartphones and satellite imagery analysis. It aids in mapping the Earth's surface, predicting and mitigating disasters, locating and characterizing space objects, and contributing to space debris cleanup efforts.

\n\n
\n","metadata":{}},{"cell_type":"markdown","source":"

Conclusion

\n\n
\n

\nComputer vision is a rapidly growing field with a wide range of applications. It has the potential to revolutionize many industries, and it is already being used in a variety of ways. However, there are still many challenges that need to be addressed before computer vision can reach its full potential.\n

\n \n

\nOne of the biggest challenges facing computer vision is the lack of data. In order to train accurate computer vision models, we need large datasets of labeled images and videos. However, these datasets can be expensive and time-consuming to collect.\n

\n\n

\nAnother challenge facing computer vision is the variability of real-world data. The conditions under which images and videos are captured can vary greatly, and this can make it difficult for computer vision models to generalize to new situations.\n

\n\n

\nDespite these challenges, computer vision is a field with a lot of promise. As the field continues to develop, we can expect to see even more amazing applications of computer vision in the years to come.\n

\n\n
","metadata":{}},{"cell_type":"markdown","source":"

References and Citations

\n\n
\n \n

\n1] What is the difference between an image and a video frame? by Leonardo Chiariglione (link: https://blog.chiariglione.org/what-is-the-difference-between-an-image-and-a-video-frame/)

\n\n

\n2] What is Computer Vision? (link: https://www.ibm.com/topics/computer-vision)

\n\n

\n3] [Introduction to computer vision: History and applications (link: https://www.superannotate.com/blog/introduction-to-computer-vision)

\n\n

\n4]13 Computer Vision Algorithms You Should Know (link: https://datagen.tech/guides/computer-vision/algorithms/)

\n\n

\n5] Which algorithms are based on CNN? (prompt to Bard)

\n\n

\n6] Convolutional Neural Network - A brief history of their evolution by Brajesh Kumar (link: https://medium.com/appyhigh-technology-blog/convolutional-neural-networks-a-brief-history-of-their-evolution-ee3405568597)

\n\n

\n7] How human store images in their brain? (prompt to ChatGPT)

\n\n

\n8] How Do Computers Store Images? by Himanshi Singh (link: https://www.analyticsvidhya.com/blog/2021/03/grayscale-and-rgb-format-for-storing-images/)

\n \n

\n9] [codeburst.io](https://codeburst.io/how-are-images-stored-on-a-computer-353ac16b6d8f) How Do Computers Store Images? by Greg

\n\n

\n10] [University of Michigan Library](https://guides.lib.umich.edu/c.php?g=282942&p=1885348) Common Image File Formats

\n\n

\n11] Prompt to ChatGPT -> What really Computer Vision is?

\n\n

\n12] [Clickworker](https://www.clickworker.com/ai-glossary/computer-vision/) AI Grocerry -> Computer Vision

\n \n

\n13] [Pix4D Documentation](https://support.pix4d.com/hc/en-us/articles/202557289-System-requirements-Minimum-and-recommended-computer-specifications) System requirements: Minimum and recommended computer specifications

\n\n

\n14] Prompt to ChatGPT -> hardware requirements for computer vision

\n\n

\n15] [Kaggle-Computer-Vision-Competitions-List](https://github.com/mrgloom/Kaggle-Computer-Vision-Competitions-List)

\n\n

\n16][ Computer vision and cloud computing](https://www.linkedin.com/pulse/computer-vision-cloud-computing-yash-tavse/) by Yash Tavse

\n\n

\n17] [Computer Vision Trends That will Dominate the Industry in 2023](https://analyticsindiamag.com/computer-vision-trends-that-will-dominate-the-industry-in-2023/) By Mohit Pandey

\n\n

\n18] [Deep Learning vs. Traditional Computer Vision](https://arxiv.org/ftp/arxiv/papers/1910/1910.13796.pdf#:~:text=Before%20the%20emergence%20of%20DL,be%20involved%20in%20this%20step.) by Niall O’ Mahony, Sean Campbell, Anderson Carvalho, Suman Harapanahalli,\nGustavo Velasco Hernandez, Lenka Krpalkova, Daniel Riordan, Joseph Walsh

\n\n

\n19] [CLASSIC MACHINE VISION VS. DEEP LEARNING](https://www.mvtec.com/technologies/deep-learning/classic-machine-vision-vs-deep-learning) by MVtech software

\n\n

\n20] [Deep Learning Vs. Traditional Computer Vision](https://naadispeaks.blog/2018/08/12/deep-learning-vs-traditional-computer-vision/) by NaddiSpeaks

\n\n

\n21] Prompt to ChatGPT - list out all algorithms names of computer vision, it should include both deep learning as well as traditional algorithms

\n\n

\n22] [A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way](https://saturncloud.io/blog/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way/)\n by SaturnCloud

\n \n

\n23] [Introduction to Convolutional Neural Networks (CNN)](https://www.analyticsvidhya.com/blog/2021/05/convolutional-neural-networks-cnn/) by Analytis Vidya

\n\n

\n24] [Convolutional Neural Networks, Explained](https://towardsdatascience.com/convolutional-neural-networks-explained-9cc5188c4939) by Mayank Mishra from towardsdatascience blog

\n\n

\n25] [Convolution](https://en.wikipedia.org/wiki/Convolution#:~:text=In%20mathematics%20(in%20particular%2C%20functional,the%20process%20of%20computing%20it.) from Wikipedia

\n\n

\n26] [Activation functions in Neural Networks](https://www.geeksforgeeks.org/activation-functions-neural-networks/) by Sakshi Tiwari from GeeksForGeeks

\n\n

\n27] [Activation Functions in Neural Networks](https://towardsdatascience.com/activation-functions-neural-networks-1cbd9f8d91d6) by Sagar Sharma from towardsdatascience

\n \n

\n28] [What Is Computer Vision? [Basic Tasks & Techniques]](https://www.v7labs.com/blog/what-is-computer-vision) by \nHmrishav Bandyopadhyay from V7Labs

\n \n

\n29] [Different Computer Vision Tasks](https://ananya-banerjee.medium.com/different-computer-vision-tasks-b3b49bbae891) by Ananya Banerjee from medium\n

\n \n

\n30] [Image detection, recognition and image classification with machine learning](https://medium.com/ai-techsystems/image-detection-recognition-and-image-classification-with-machine-learning-92226ea5f595) by Renuka Soni from medium\n

\n \n

\n31] [Computer Vision - Face Detection] by Jasmin Kurtanović ( https://serengetitech.com/tech/computer-vision-face-detection/ )\n

\n \n

\n32]Prompt to Bard -> tell me in detailed about pre-trained models and its impact on computer vision. Then tell me about recentaly most use pre-train models in computer vision. Then tell me significance of pre-trained models. Give me benefits, definitions, working, significance of pre-trained models in computer vision\n

\n

33] Imagen https://imagen.research.google/

\n

34] Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding\n by Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, Mohammad Norouzi link- https://arxiv.org/abs/2205.11487

\n
\n\n","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd","metadata":{"execution":{"iopub.status.busy":"2023-07-13T16:08:21.805878Z","iopub.execute_input":"2023-07-13T16:08:21.806252Z","iopub.status.idle":"2023-07-13T16:08:21.812298Z","shell.execute_reply.started":"2023-07-13T16:08:21.806222Z","shell.execute_reply":"2023-07-13T16:08:21.810551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = pd.read_csv(\"/kaggle/input/2023-kaggle-ai-report/sample_submission.csv\")\ndata","metadata":{"execution":{"iopub.status.busy":"2023-07-13T16:08:22.842029Z","iopub.execute_input":"2023-07-13T16:08:22.842477Z","iopub.status.idle":"2023-07-13T16:08:22.860009Z","shell.execute_reply.started":"2023-07-13T16:08:22.842444Z","shell.execute_reply":"2023-07-13T16:08:22.859000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nsubmission = pd.DataFrame({\n \"type\" : [\"essay_category\", \"essay_url\",\"feedback1_url\",\"feedback2_url\", \"feedback3_url\"],\n \"value\" : [\n \"Image and/or video data\", \n \"https://www.kaggle.com/prathmeshadsod/dive-in-computer-vision/\", \n 'https://www.kaggle.com/code/sohamjangra/generative-ai-a-revolution-in-ai/comments#2342050',\n 'https://www.kaggle.com/code/ethanhu8093/meta-analysis-on-kaggle-computer-vision/comments#2342125',\n 'https://www.kaggle.com/code/astergalemayehu/ai-ethics-for-beginners/comments#2343355'\n ],\n }\n)\n\nsubmission\n","metadata":{"execution":{"iopub.status.busy":"2023-07-13T16:08:23.849711Z","iopub.execute_input":"2023-07-13T16:08:23.850121Z","iopub.status.idle":"2023-07-13T16:08:23.862393Z","shell.execute_reply.started":"2023-07-13T16:08:23.850091Z","shell.execute_reply":"2023-07-13T16:08:23.860518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2023-07-13T16:08:24.960774Z","iopub.execute_input":"2023-07-13T16:08:24.961127Z","iopub.status.idle":"2023-07-13T16:08:24.967303Z","shell.execute_reply.started":"2023-07-13T16:08:24.961101Z","shell.execute_reply":"2023-07-13T16:08:24.966286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.read_csv(\"submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-07-13T16:08:26.479023Z","iopub.execute_input":"2023-07-13T16:08:26.479472Z","iopub.status.idle":"2023-07-13T16:08:26.493486Z","shell.execute_reply.started":"2023-07-13T16:08:26.479437Z","shell.execute_reply":"2023-07-13T16:08:26.491921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}