{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Unveiling the Evolution of Generative AI: Insights, Advancements, and Lessons Learned in Analytics💡\nBy Suman Adhikari\n\n# Abstract\nRohit Ghai, chief executive officer of RSA Security, said, \"Every new technology wave is bigger, faster, and more disruptive than all previous ones. This time is no different.\"  Currently, as shown in Figure 1, there is a shift towards Generative Artificial Intelligence (AI) and data-driven models, leading to concerns about the security implications. Over the past two years, technology has rapidly evolved in AI and machine learning, creating numerous generative AI applications. AI is now extensively employed across various sectors. From its early conceptual beginnings to recent breakthroughs driven by machine learning and big data, AI has evolved into a transformative technology with wide-ranging applications. This essay covers what the community has learned over the past two years of working and experimenting with generative AI. This essay explores the advancements and insights gained from working and experimenting with generative models in analytics over the past two years. Generative models, such as variational autoencoders and generative adversarial networks, have revolutionized data analysis and synthesis by creating new data samples that closely resemble the training data. The essay highlights the key techniques and algorithms used in generative modeling. It discusses their applications across domains, including image synthesis, text generation, and healthcare. It analyzes the major advancements and breakthroughs in generative modeling, presenting specific examples and case studies to illustrate their practical implications. The essay also addresses the lessons learned by the community, familiar challenges encountered, and the approaches employed to overcome these challenges. It assesses the impact of generative models on the field of analytics and related domains, highlighting their potential to drive innovation and provide valuable insights. Ethical considerations, including biases, fairness, and transparency issues, are discussed, emphasizing the importance of responsible use and addressing these concerns. \n","metadata":{}},{"cell_type":"markdown","source":"\n<p style=\"text-align:center;\">\n  <img src=\"https://drive.google.com/uc?id=1HJs7HQM6njDTy5ZKQBf71_zeyzEroef9\" alt=\"Generative AI Startups and Valuations\" style=\"width:800px;\"/>\n</p>\n<p style=\"text-align:center;\">\n  <strong>Figure 1:</strong>  Generative AI Startups and Valuations. From Generative AI is churning out unicorns like it’s 2021, by CBINSIGHTS, 2023.\n</p>\n","metadata":{}},{"cell_type":"markdown","source":"\n# Understanding the Historical Context of AI\nAI has a rich and fascinating history that spans several decades, evolving from a science fiction notion to a transformative technology with wide-ranging applications. In this introduction, the learnings and advancements in generative AI over the past two years provide a broader historical context to understand the significance and impact of recent developments. By combining historical perspective with contemporary insights, we offer a holistic view of the progress made in the broader field of AI and the specialized domain of generative AI. This knowledge is a foundation for exploring the insights, challenges, and advancements achieved over the past two years. Through this exploration, we look to contribute to the collective understanding of generative AI and pave the way for future improvements in this exciting and rapidly evolving field.\nThe journey of AI began in the mid-20th century when pioneers like Alan Turing posed the fundamental question of whether machines have intelligence (McCarthy et al., 2006). This inquiry sparked breakthroughs, setbacks, and later revivals that shaped the trajectory of AI research. Over time, computers became faster and more powerful, algorithms became sophisticated, and the availability of vast amounts of data propelled AI to new heights. Important dates and success stories mark the history of AI, including the 1956 Dartmouth Summer Research Project on AI, which set up AI as a field of study. Early successes in the late 1950s to early 1970s, such as the development of the Logic Theorist and Joseph Weizenbaum's ELIZA, highlighted promising problem-solving and language interpretation capabilities. Significant milestones like IBM's Deep Blue defeating world chess champion Gary Kasparov in 1997 proved AI's potential in complex decision-making tasks. The 2000s saw advancements in natural language processing, image recognition, and emotionally expressive robots like Cynthia Breazeal's Kismet. These achievements and breakthroughs in machine learning and big data have propelled AI to prominence across various industries (Smith et al., 2006). Figure 2 shows a timeline of important dates and milestones in the history of AI.\n","metadata":{}},{"cell_type":"markdown","source":"\n\n<p style=\"text-align:center;\">\n  <img src=\"https://drive.google.com/uc?id=1gIUOfftZXL9x8Zv5vgA1yukmhaNPdSYa\" alt=\" AI Timeline\" style=\"width:1000px;\"/>\n</p>\n<p style=\"text-align:center;\">\n  <strong>Figure 2:</strong>  AI Timeline. From Express Writers: AI in Content Marketing: How Will it Impact Content in 2023?, by  Adam Oakley, 2023.\n</p>","metadata":{}},{"cell_type":"markdown","source":"# The Rise of Generative AI: Concepts and Distinctions\nIn today's rapidly evolving landscape of analytics, one area that has captured the imagination and sparked significant advancements is generative AI. Generative models have unlocked new insights, enabled realistic simulations, and driven innovation across industries. Imagine a machine capable of analyzing vast amounts of data and creating new samples that closely resemble the training data. Over the past two years, the community of researchers, data scientists, and innovators has been diligently working and experimenting with generative AI, pushing the boundaries of what is possible. Generative models are machine learning models that aim to understand and mimic the underlying probability distribution of the data they are trained on (Goodfellow et al., 2020). Generative models generate new samples that resemble the training data, allowing them to create original content or generate realistic simulations. Unlike discriminative models that focus on modeling the conditional probability of a target variable given input data, generative models aim to capture the underlying patterns and structure of the entire dataset by modeling the joint probability distribution of both the target variable and the input data. As explained in Bishop's book on pattern recognition and machine learning, generative models go beyond predicting specific target variables and instead seek to understand the holistic nature of the data (Bishop, 2006). Generative models are used for various tasks, including data synthesis, augmentation, anomaly detection, and generating new samples. As explained in Goodfellow's book, generative models can produce new instances with similar characteristics to the training data by understanding the underlying data distribution. This ability to generate new samples is beneficial in scenarios with limited data availability or when new data is desired (Goodfellow. 2016).\nOn the other hand, Discriminative models are primarily focused on learning the boundary or decision boundary between different classes or categories in the data. They are designed to classify or discriminate between different instances based on their input features. Discriminative models are often used for tasks such as classification and regression, emphasizing predicting the target variable given the input data (Murphy, 2012).\n\n# Techniques and Algorithms in Generative AI\nTo understand the work and experiment in generative AI, we must understand the concept and history of generative modeling. The advancement of these techniques and algorithms in generative modeling has paved the way for developing the most recent and powerful generative AIs. Generative modeling encompasses a range of techniques and algorithms; two of the most prominent are transformer-based language models and generative adversarial networks (GANs). These approaches have significantly advanced the field of generative modeling and have been widely adopted for various applications.\n\n- **Transformer-Based Language Models:**\n    A Transformer-Based Language Model is a neural network that learns context and meaning by tracking its relationships in sequential data like the words in the sentence. Positional encoders are used in Transformers to assign tags to data elements entering and exiting the network. These tags enable attention units to examine the relationships between elements and generate an algebraic map representing these connections. Multi-headed attention involves executing attention queries in parallel by computing a matrix of equations. This approach enhances the model's ability to capture diverse patterns and dependencies within the data. Using these mechanisms, computers can perceive patterns like humans, enabling them to identify and understand complex relationships and structures within the input data. This ability to recognize patterns helps various tasks, such as natural language understanding, image recognition, and other forms of data analysis. First described in a 2017 paper from Google, Transformer based language models are the most prominent models invented to date. In 2023, they have gained considerable interest from researchers, developers, and the broader AI community.\n\n- **Generative Adversarial Networks (GANs):**\n  GANs are a framework that involves training two neural networks simultaneously: a generator and discriminator networks. The generator network, typically a convolutional neural network, takes random noise as input and generates synthetic data samples. In contrast, the discriminator network, typically a deconvolutional neural network, aims to distinguish between real and generated data. The training process involves an adversarial game, where the generator aims to generate realistic samples to deceive the discriminator. In contrast, the discriminator aims to classify between real and generated data accurately. Through this iterative process, both networks improve their performance. GANs have successfully generated highly realistic and diverse samples, such as images, by capturing intricate patterns and structures in the training data (Goodfellow et al., 2020). Nvidia uses GANs model for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs) (Isola et al., 2018).\n  \n- **Variational Autoencoders (VAEs):**\n  The VAE framework, introduced in the paper by Kingma and Welling (2013), has been widely adopted and applied in various domains. VAEs are probabilistic generative models that combine the concepts of autoencoders and variational inference. An autoencoder consists of an encoder network that maps input data to a lower-dimensional latent space representation and a decoder network that reconstructs the input data from the latent space. VAEs introduce probabilistic modeling to this framework, treating the latent space as a probability distribution rather than a fixed encoding. VAEs optimize a loss function that encourages the latent space to follow a prior distribution (usually a Gaussian distribution) and the reconstructed data to match the original input. By sampling from the learned latent space, VAEs can generate new data samples with similar characteristics to the training data (Kingma & Welling, 2013).\n\n- **Autoregressive Models:**\n    Autoregressive models are another class of generative models that model the joint probability distribution of the data by assuming a sequential generation process. These models predict the probability of each element in the data based on the previously generated elements. Popular examples of autoregressive models include PixelCNN and WaveNet, which have achieved impressive results in generating images and audio, respectively.\n\n- **Flow-Based Models:**\n  Flow-based models approach generative modeling by modeling the transformation process from a simple base distribution to the target data distribution. They learn a series of invertible transformations that map samples from the base distribution to the target distribution. Flow-based models can generate high-quality samples and provide efficient density estimation. Flow-based models are used in various tasks like Audio generation (Ping et al., 2020), image generation (Kingma & Dhariwal, 2018), video generation (Kumar et al., 2019), and anomaly detection (Rudolph et al., 2021).\n\nThese are just a few examples of the traditional and advanced techniques and algorithms used in generative modeling. Each approach has its strengths and limitations, and the choice of algorithm depends on the specific task and requirements. The field of generative modeling is dynamic, and researchers continue to explore new techniques and variations to improve the quality and diversity of generated samples. The later section of the essay will focus on the most advanced model and technologies used in generative AI.\n\n\n# Notable Applications of Generative Models Across Different Domains\n\nGenerative models have found diverse applications across various domains, highlighting their versatility and potential. The impact of generative AI models on various domains is shown in Figure 3.\n\n<div align=\"center\">\n    <figure>\n        <img src=\"https://drive.google.com/uc?id=1nLI5og64LOf2a4zVnwd9OHeJWYehq2K4\" alt=\"The impact of Generative AI on secotrs\" style=\"width:600px;\">\n        <p style=\"text-align:center;\">\n        <strong>Figure 3:</strong> The impact of Generative AI on sectors. From HSBC: Generative AI-- The rise of the machines, by  Mark McDonald, 2023.\n        </p>\n    </figure>\n</div>\n\n- **Image Synthesis:** Generative models, particularly GANs, have been extensively used for image synthesis tasks. They can generate realistic images that resemble the training data distribution. These models have been applied to tasks such as image super-resolution, style transfer, image inpainting (filling in missing parts of an image), and generating novel and diverse images.\n\n- **Text Generation:** Generative models have significantly contributed to text generation tasks, including natural language generation, dialogue systems, and machine translation. Recurrent Neural Networks (RNNs) and variants like Long Short-Term Memory (LSTM) networks have been employed to generate coherent and contextually relevant text. Language models such as OpenAI's GPT (Generative Pre-trained Transformer) have proven impressive capabilities in generating human-like text.\n\n- **Music Composition:** Generative models have been used to compose original music. By training on large datasets of musical compositions, recurrent neural networks or transformer models can learn the patterns and structures of music and generate new compositions. These models can create melodies, harmonies, and even complete musical arrangements. Companies like TensorFlow have used autoregressive models to generate high-quality music (Hawthorne et al., 2022).\n\n- **Video Generation:** Generative models have been applied to video generation tasks, such as generating new video sequences or changing existing ones. Generative models can generate realistic and coherent video sequences by modeling videos' temporal dependencies and dynamics. This has applications in video editing, content creation, and special effects.\n\n- **Anomaly Detection:** Generative models can identify anomalies or outliers in datasets by modeling the normal data distribution. They can detect deviations from the learned patterns, enabling anomaly detection in various domains, including fraud detection and cybersecurity.\n\nThese applications prove the wide range of possibilities offered by generative models, paving the way for innovation and advancements in various industries and domains.\n\n","metadata":{}},{"cell_type":"markdown","source":"# Recent Advances in Generative AI\nGenerative AI has seen significant advancements in recent years, pushing the boundaries of what is possible in generating realistic and creative outputs. One popular and advanced model is GPT-3 (Generative Pre-trained Transformer 3), a Large Language Model (LLM), and its successor, GPT-4, is a large multi-modal model that accepts both image and text inputs and generates text outputs. GPT-4 pushes the boundaries by using the success of LLMs in single domains, such as computer vision and natural language processing, and extends it to understand complex mathematical and physical diagrams by recognizing unusual image patterns. An LLM is a type of generative AI since they generate novel combinations of text in the form of natural-sounding language.\nChatGPT, an AI chatbot developed by OpenAI, is one specific application of Transformer-based language models. It gained significant attention and popularity, with one million users in the first five days after its release. According to UBS research, by January 2023, it had reached one hundred million monthly active users, making ChatGPT the fastest-growing consumer application in history. Here are some notable recent advances in generative modeling:\n\n- **Large Language Model Meta AI (LLaMA):** LLaMA is a LLM released by Meta AI in February 2023. They trained a variety of model sizes ranging from seven billion to sixty-five billion parameters. LLaMA uses the transformer architecture. It has been the standard architecture for language modeling since 2018. The Stanford University Institute for Human-Centered AI (HAI) Center for Research on Foundation Models (CRFM) released Alpaca. Alpaca is a training recipe based on the LLaMA 7B model that uses the \"Self-Instruct\" method of instruction tuning to acquire capabilities (Wang et al., 2022) (Liu et al., 2023). According to medium.com, LLaMa is trained in diverse texts, including facts, news, scientific articles, and more. In contrast, ChatGPT is trained primarily on internet text, such as web pages and social media posts. LLaMa and ChatGPT are both highly advanced language models that have the potential to revolutionize the way we use natural language processing.\n\n- **Progressive Growing of GANs:** GANs are one of the most powerful models in AI image generation. This technique involves progressively training GANs, starting from low-resolution images and gradually increasing the complexity. It allows for generating high-quality, high-resolution images with improved stability and diversity. Manu et al. (2023) research shows that GANs have significantly impacted image and text generation and drug discovery by generating realistic molecular structures (Manu et al., 2023).\n\n- **StyleGAN:** StyleGAN introduced a novel approach for controlling the style and attributes of generated images. It allows users to manipulate specific characteristics of generated images, such as facial expressions or hairstyles, by disentangling the latent space representation. StyleGAN2 enhances the quality and control of generated images. It improves the original StyleGAN by addressing artifacts, improving training efficiency, and introducing novel techniques for fine-grained control over image synthesis. StyleGAN3 improves upon StyleGAN2 by solving the \"texture sticking\" problem. It can generate images that solve texture sticking, rotate, and translate easily (Bermano et al., 2022).\n\n- **Security Improvements:** Generative AI is used in the security field to make systems more secure and help the SOC team summarize emerging information, explain remediation options, and automate the work needed to neutralize threats during the incident response process. For example, Microsoft has unveiled its new security product called Microsoft Security Copilot, which uses its proprietary security-based model and OpenAI’s GPT-4 generative AI model to identify breaches, connect threat signals and analyze data to improve threat detection tactics, and speed up incident response process.\n\nThese recent advances in generative modeling have expanded the capabilities of AI systems in generating realistic images, text, and other creative outputs. They have paved the way for exciting art, design, entertainment, and virtual reality applications. They continue to drive innovation in generative AI.","metadata":{}},{"cell_type":"markdown","source":"# Emerging Trends and Market Insights in Generative AI\n\n- As of June 2023, McKinsey, a global consulting company with approximately 30,000 employees across 67 countries, stated that almost 50% (half of) its employees use ChatGPT and other such generative AI tools.\n- In its recent survey, Altman Solon stated that one in every four companies in the United States is utilizing generative AI tools.\n- According to the State of AI report, by September 2023, generative audio tools are expected to attract over 1,00,000 developers.\n- Gartner stated in its report on generative AI that by 2025, approximately 30% of newly discovered drugs will be found with the help of AI tools.\n- China's search engine Baidu announced to invest approximately 1 billion Yuan ($140 million) in nurturing its interest in AI self-reliance.\n- Micron Technology announced investing up to USD 3.6 billion in Japan, along with close support from the Japanese government; the massive investment will focus on the innovation of generative AI chips in Japan.\n- In April 2023, the prime Minister of Japan stated that the country is openly supporting the use of industrial generative AI such as ChatGPT techniques.\n- In April 2023, Pwc announced investing over USD 1 billion to expand the scale of generative AI; the company has planned to showcase this investment for the next three years.\n- By March 2023, Microsoft has already invested USD 13 billion in OpenAI\n\n\n# Lessons Learned from Working with Generative AI - Challenges and Limitations\nThrough working and experimenting with generative models over the past two years, the community has gained valuable insights and lessons that have shaped the field of generative modeling. Here are some key lessons learned:\n\n- **Model Robustness and Generalization:** It has become clear that ensuring the robustness and generalization of generative models is crucial. Overfitting the training data can lead to unrealistic or biased outputs. Overfitting occurs when the model cannot generalize and fits too closely to the training dataset. Overfitting occurs if the training data size is too small and needs more data samples to predict and accurately represent all possible input values if there is too much noisy data and the model complexity is high. An overfit model can give inaccurate predictions. The quality and quantity of training data directly affect the performance of generative models. Insufficient or biased training data can limit the model's ability to generate high-quality outputs. The community has learned the importance of collecting diverse and representative datasets to capture the full range of variations and complexities in the target domain. Regularization, various training data, and augmentation strategies are essential to improve model performance and generalization.\n\n- **Ethical Considerations:** The advancements in generative modeling have raised ethical concerns about the potential misuse of synthetic content. The community has realized the need for responsible development and usage of generative models, emphasizing transparency, fairness, and accountability. However, several incidents have also affected the use of these limitations of generative AI. According to the Deloitte Risk and ethical considerations of Generative AI post published on March 13, 2023, the potential risk of generative AI is copyright infringement. Several companies banned generative AI content due to apprehensions about potential copyright claims. Companies like Getty Images have banned AI-generated content over fears of legal challenges. New York City public schools have banned ChatGPT due to raised risks of plagiarism from ChatGPT. Efforts are being made to address deepfakes, privacy concerns, and the potential for malicious manipulation. Various organizations have participated in developing an ethical framework for AI with five principles non-maleficence, responsibility or accountability, transparency and explainability, justice and fairness, and respect for various human rights, such as privacy and security(Jobin, A., 2019). \n\n- **Interdisciplinary Collaboration:** The community has recognized the value of interdisciplinary collaboration in generative modeling. Experts from diverse fields, including computer science, mathematics, psychology, and domain-specific knowledge, must work together to understand the unique challenges and opportunities in different domains and develop robust and effective generative models. As an example of massive interdisciplinary collaboration, MIT is actively working to rectify naive misconceptions and off-key research directions, informing policy decisions. They are looking into four areas, like the societal impact of generative AI and how to center human agency in each: the economics of the creative industry, attribution ownership, aesthetics, and news.\n\n- **Iterative Improvement:** Continuous iteration and refinement are critical in developing generative models. The community has learned that the iterative process of experimenting, evaluating, and incorporating feedback leads to significant improvements in model performance and capability. Iterative approaches allow for fine-tuning models, exploring new techniques, and addressing limitations. As discussed earlier, LLaMA developers' iterative and continuous improvements in the generative model created the most performant LLM for researchers and noncommercial use cases. Meta released LLaMa in February 2023. LLaMa is a more parameter-efficient, open alternative to large commercial LLMs. So many improvements have been made in the generative model, leading us to the most recent advancement in generative AI.\n\n- **Domain-Specific Adaptation:** Generative models often require adaptation to specific domains to achieve the best results. The community has learned that understanding the underlying data distribution and task-specific constraints, which are part of domain-specific knowledge, is crucial for developing tailored generative models that effectively address the challenges and requirements of the target domain.DALL-E2 and ChatGPT are the two perfect examples of domain-specific research that Open Ai is adapting. DALLE can create realistic images and art, and ChatGPT for giving articulated and detailed responses to the questions.\n\n- **Real-World Validation and User Feedback:** The community has recognized the importance of confirming generative models in real-world scenarios and gathering user feedback. Real-world validation helps find limitations, assess practical usability, and understand the impact of generative models on end-users. User feedback supplies insights into user preferences, concerns, and potential improvements. According to Forbes, AI lacks self-awareness and is unaware of its limitations. It makes decisions without recognizing the potential risk and shows a massive scale of \"unconscious incompetence.\" Therefore, real-world validation and human user feedback are essential to the language models and the generative AI algorithms they feed.\n\nThese lessons highlight the ongoing progress and evolving understanding within the generative modeling community. They guide future research and development, emphasizing the need for robustness, ethical considerations, interdisciplinary collaboration, and user-centric approaches to further advance the field of generative modeling.\n\n\n# Future Directions and Advancements in Generative Modeling\nBased on current trends and challenges, generative modeling is poised for exciting advancements and developments. Here are some potential future directions based on current trends and challenges:\n\n- **Improved Model Training and Stability:** One major challenge in generative AI is ensuring stable and reliable model training. Future advancements may focus on developing more robust training algorithms and architectures that mitigate mode collapse and training instability. Techniques like regularization, novel loss functions, and architecture modifications may contribute to more stable and dependable generative models. Regularization can be used to calibrate machine learning models to prevent overfitting or Underfitting because the side training dataset is not enough, the model is too simple, the training data is not cleaned and contains noise, and the model is too complex. Below figure 4 shows how data are fitted in Underfitting and overfitting models.\n\n\n","metadata":{}},{"cell_type":"markdown","source":"\n<p style=\"text-align:center;\">\n  <img src=\"https://drive.google.com/uc?id=1C0oggl2B7Wo1yYGBTWxK8zItImaszjXk\" alt=\"Underfitting and Overfitting\" style=\"width:800px;\"/>\n</p>\n<p style=\"text-align:center;\">\n  <strong>Figure 4:</strong>  Underfitting and Overfitting. From GeeksforGeeks: ML | Underfitting and Overfitting, by Dewang Nautiyal\n</p>","metadata":{}},{"cell_type":"markdown","source":"- **Enhanced Diversity and Controllability:** Generating diverse and controllable outputs is another area of focus. Generative Ai depends upon a large amount of data, which should be diverse and unbiased, so the output of the generative data is representative and equitable. Future research should explore techniques to improve the diversity of generated samples, allowing generative models to produce a wider range of realistic and novel outputs. Moreover, efforts should be directed towards developing fine-grained control over generated samples, enabling users to specify desired attributes or characteristics in the generated outcomes.\n\n- **Domain-Specific Applications, Cross-Domain, and Multi-Modal Generation:** Generative AI has advanced in several domains. For example, generative AI aids in medical imaging analysis, drug discovery, and personalized medicine in healthcare. Similarly, generative AI is used in robotics for motion planning, grasping, and manipulation tasks. Figure 5 shows that generative AI has made significant strides in various domains. Generative models have predominantly focused on single-domain tasks like image synthesis or text generation. Future directions in generative modeling should extend its capabilities to domain-specific applications and cross-domain and multi-modal settings. In these settings, models can generate outputs that combine different modalities, such as images and text or perform domain translation. Cross-domain data mapping could be the key in different domains so that generated output matches each other. This expansion can enable various applications, including forging image captions, creating art music video production, or bridging the gap between different data modalities.\n","metadata":{}},{"cell_type":"markdown","source":"\n<p style=\"text-align:center;\">\n  <img src=\"https://drive.google.com/uc?id=1laKvsDG6bw1hz8xTiZ5SaYWVhX2Wm97e\" alt=\"Generative AI Trend \" style=\"width:800px;\"/>\n</p>\n<p style=\"text-align:center;\">\n  <strong>Figure 5:</strong>  Generative AI Trend. From: Base10: The Rise of New Software Ecosystems, by TJ Nahigian & Luci Fonseca, 2022\n</p>\n","metadata":{}},{"cell_type":"markdown","source":"- **Explainability and Interpretability:** As generative models become more powerful and complex, ensuring their explainability and interpretability becomes crucial. Future research should focus on developing techniques that provide insights into the internal workings of generative models, thereby making the decision-making process of these models more transparent and representative. By achieving this, it will be possible to address concerns related to bias, fairness, and accountability in generative models.\n\n- **Few-Shot and Zero-Shot Learning:** One challenge in generative modeling is the reliance on copious amounts of labeled training data. Few-shot learning classifies new data when only a few training samples have supervised information. In contrast, Zero-shot learning allows a model to organize objects from previously unseen classes without receiving any specific training for those classes. Several research is already done to assess the accuracy, reliability, and bias of the LLM with zero-shot learning (Törnberg, P., 2023). Future advancements should explore more techniques for few-shot and zero-shot learning, wherein generative models can learn from limited or no labeled data. This approach will enable the models to adapt and generate meaningful outputs in scenarios where labeled data is scarce or unavailable.\n\n- **Security and Responsible Generative AI:** With the increasing capabilities of generative AI, ethical considerations become paramount. In the recent RSA conference 2023 held in San Francisco, Rohit Ghai, CEO of RSA, spoke about \"Good AI\" and \"Bad AI\" in his opening keynote. The threat actor will use \"Bad AI,\" and generative AI will be used in sophisticated phishing and malware creation, so security professionals should use \"Good AI\" to mitigate the threat. We should focus on developing frameworks and guidelines for generative AI's responsible and ethical use. This includes addressing issues such as potential misuse, privacy concerns and developing safeguards to prevent the malicious use of generative AI.\n\nIt is important to note that these potential future directions are speculative and subject to ongoing research and development. The field of generative AI is evolving rapidly, driven by advancements in machine learning, deep learning, and computational resources. With continued innovation and interdisciplinary collaboration, generative modeling can revolutionize various industries, inspire new creative endeavors, and further our understanding of data generation and synthesis.\n\n# Ethical Considerations \nEthical considerations and responsible use of generative models are paramount to ensure their deployment aligns with societal values and promotes fairness, transparency, and accountability. Generative AI models can inadvertently perpetuate biases in the training data, leading to biased or unfair outputs. It is crucial to identify and mitigate such biases to ensure fair and fair results. This involves carefully curating diverse and representative training data, conducting thorough bias assessments, and incorporating fairness-aware algorithms that mitigate biases in the generated outputs. Generative models are often a complex and black box, making it challenging to understand their decision-making process. Enhancing transparency and explainability is essential for building trust and understanding of the generated results. Research efforts should focus on developing techniques to interpret and explain the inner workings of generative models, allowing users to understand why certain outputs are generated and enabling the detection of any unintended biases or errors. Generative models rely on training data, and the responsible use of data is paramount. This includes ensuring proper data collection, consent, and privacy protection. Data sources should be carefully chosen to prevent the misuse or infringement of individuals' rights. Data anonymization techniques should also be employed to safeguard sensitive information and preserve privacy. Generative models are susceptible to adversarial attacks, where malicious actors manipulate inputs to produce undesired or misleading outputs. Robustness against such attacks should be a focus, with the development of defense mechanisms and adversarial training methods to enhance the security of generative models. As generative models have a significant societal impact, it becomes crucial to establish accountability and regulatory frameworks. This involves the development of clear guidelines and standards for developing, deploying, and using generative models. Collaborative efforts between researchers, policymakers, and industry stakeholders are necessary to ensure responsible practices and adherence to ethical standards. Promoting user education and awareness about generative models is vital. Users should clearly understand the capabilities and limitations of generative models to make informed decisions. Following guidelines, best practices, and educational resources can empower users to use generative models responsibly and ethically. Ethical considerations and responsible use of generative models should be an ongoing process. Continuous monitoring and evaluation of the model's performance, impact, and potential biases are necessary to identify and address emerging ethical concerns. Regular audits and assessments can ensure adherence to ethical guidelines and help improvements in the technology. Addressing these ethical considerations and promoting the responsible use of generative models will foster trust, mitigate potential harms, and support the development of inclusive and beneficial applications across various domains. Collaboration between researchers, policymakers, industry stakeholders, and the wider public is essential to navigate the ethical challenges and shape the future of generative modeling in a responsible and accountable manner.\n\n# The Role of Human Analysis in Contrast to Generative AI\nGenerative AI is good at processing data but needs to improve at thinking in the abstract. Humans analyze different aspects and perspectives, taking into account factors such as a person's background, the emotions of the audience or listener, the potential impact on individuals, and the possible consequences of their words or actions. They consider these elements when delivering a conversation, writing a letter, giving a presentation, etc. Humans can sense emotions and can adjust their approach if they feel something is not appropriate or if it may have negative consequences. Based on their situation analysis, they can even interrupt or change direction during a conversation.\nOn the other hand, generative AI is motivated to produce an output that satisfies a specific objective or goal. It does not possess the same analytical capabilities as humans. While humans can evaluate and measure things twice, allowing them to select the most appropriate option, generative AI lacks this ability. It produces an output based on predetermined algorithms and may possess a different level of discernment and critical thinking than humans. This difference in analytical capabilities and decision-making processes is one of the reasons why companies like Microsoft and GitHub refer to generative AI as a 'copilot' rather than a 'pilot.' It highlights the collaborative nature of generative AI, where it assists and augments human expertise rather than replacing it entirely.\n","metadata":{}},{"cell_type":"markdown","source":"<div style=\"display:flex; justify-content:center;\">\n  <div style=\"width: 50%; padding: 30px; border: 1px solid black;\">\n    <img src=\"https://drive.google.com/uc?id=10PRFPw1-4lwZnE5g3opP_Pov6J4XlAyx\" alt=\"Generative AI\" style=\"width: 100%; height: 100%;\">\n      <p style=\"text-align: center;\"><strong>Figure 6:</strong> Generative AI Brain. From aroundkent magazine: Computational Thinking, by Paul Wang</p>\n    </div>\n  <div style=\"width: 50%; padding: 30px; border: 1px solid black;\">\n    <img src=\"https://drive.google.com/uc?id=1NgM2xbxdTxT1fQXwpXdKSyLYDqhNzhMa\" alt=\"Human Brain\" style=\"width: 100%; height: 100%;\">\n     <p style=\"text-align: center;\"><strong>Figure 7:</strong> Human Brain. From elifescience.org: Linking brain structure to behavior, by elife, 2019</p>\n    </div>\n\n","metadata":{}},{"cell_type":"markdown","source":"\n# How Generative AI Will Change Our Lives Forever\nStack overflow surveys 2023 shows that while researching new tools and technologies, 15.39% of all respondents use generative AI tools to evaluate new tools, which is just the beginning. There are several things to come, and at some point, Synthetic Media will become the norm, impersonation will become a major problem, generative AI-powered work apps will help increase productivity, and AI search engines will deliver instant answers like ChatGPT, Bing AI, BARD, etc. ","metadata":{}},{"cell_type":"markdown","source":"<p style=\"text-align:center;\">\n  <img src=\"https://drive.google.com/uc?id=1hLdSU4azHSumqu0JfUFanqfh-z0gHjFh\" alt=\"AI search Engine \" style=\"width:800px;\"/>\n</p>\n<p style=\"text-align:center;\">\n  <strong>Figure 8:</strong>  AI search Engine. From makeusof.com: ChatGPT vs. Microsoft Bing AI vs. Google Bard: Which Is the Best AI Chatbot?,by MAXWELL TIMOTHY ,2023\n</p>\n\n","metadata":{}},{"cell_type":"markdown","source":"**Business and Opportunities:** According to Business Wire, A Berkshire Hathaway company and Acumen Research and Consulting, the global generative AI market was valued at USD 7.6 billion in 2021 and is poised to grow from USD 10.16 billion in 2022 to USD 110.8 billion by 2030, growing at a CAGR of 34.3% in the forecast period (2023-2030). Companies like NVIDIA and Microsoft are increasing their R&D expenditures to advance in chip making and d hardware improvement to accelerate AI calculations. During his keynote address kicking off COMPUTEX 2023, NVIDIA founder and CEO Jensen Huang introduced a new generative AI to support game development, NVIDIA Avatar Cloud Engine (ACE) for Games.","metadata":{}},{"cell_type":"markdown","source":"<p style=\"text-align:center;\">\n  <img src=\"https://drive.google.com/uc?id=1vrePGKGm_TNrCyyrWHSZjw1hQcq8M8XK\" alt=\"Generative AI Market Growth 2018- 2030 \" style=\"width:800px;\"/>\n</p>\n<p style=\"text-align:center;\">\n  <strong>Figure 9:</strong>  AI search EngineGenerative AI Market Growth 2018- 2030. From Acumen: Generative AI Market Size to Reach USD 110.8 Billion by 2030, by Acumen Research and Consulting, 2023 .\n</p>\n","metadata":{}},{"cell_type":"markdown","source":"**Daily lives:**\nGenerative AI could change our lives and work, reducing the digital divide. It will enable easier access to learning materials and validation of information, even in areas where such resources may be limited. An illustrative example is when individuals encounter challenges in their daily lives, such as fixing a car's air conditioning system. For instance, if someone owns a Nissan 2010 and needs to repair their AC but lacks knowledge about the required parts and associated costs, generative AI tools like ChatGPT can rescue them.\n\nBy leveraging generative AI, individuals can interact with the system, providing details about their car model and their specific problem. The AI model can then analyze the input, consult its vast knowledge base, and generate accurate and relevant information about the parts needed and the estimated cost of the repair. This can empower individuals with limited automotive knowledge to make informed decisions and validate the information provided by mechanics or service centers.\n\nThis example demonstrates how generative AI can bridge the digital divide by providing accessible and reliable information to individuals who may not have easy access to expert advice or technical resources. It showcases the potential of generative AI in empowering people with the knowledge they need to address everyday challenges effectively. The link below shows how individuals without technical expertise can utilize productive AI tools like ChatGPT to access valuable advice and guidance.","metadata":{}},{"cell_type":"markdown","source":"[Example of how non-technical individuals can use generative AI tools like ChatGPT for advice](https://chat.openai.com/share/92275536-fcaf-4c54-a19f-638c578a7894)","metadata":{}},{"cell_type":"markdown","source":"# Conclusion\nOver the past two years, this essay has explored the advancements and insights gained from working and experimenting with generative models. We have discussed the key techniques, such as variational autoencoders and generative adversarial networks, and highlighted their applications across domains, including image synthesis, text generation, and healthcare. The major advancements in generative modeling have led to realistic and high-quality output, improved training stability, and enhanced model interpretability. Generative AI has emerged as a powerful tool in analytics, enabling the creation of new data samples and expanding our understanding of complex data distributions. They have revolutionized how we approach data analysis and synthesis, offering a gateway to generating original content that closely resembles the training data. Their role in analytics is evolving, and they are increasingly used in decision-making processes, creative industries, and research endeavors.\n\nContinued research, collaboration, and knowledge sharing are paramount in the generative modeling community. The field is dynamic and constantly evolving, with new challenges and opportunities. Researchers and practitioners must remain committed to pushing the boundaries of generative modeling, refining existing techniques, and developing novel approaches. Collaboration among experts from diverse disciplines will foster innovation and help interdisciplinary applications.\n\nMoreover, knowledge sharing is essential for the growth and progress of the generative modeling community. Sharing research findings, best practices, and open-source resources ensures the dissemination of knowledge and accelerates advancements. Ethical considerations and responsible use of generative models are crucial to address potential biases, ensure transparency and explainability, protect privacy, and prevent malicious misuse. Collaboration between researchers, policymakers, industry stakeholders, and the wider public is essential to establish guidelines, standards, and regulatory frameworks that promote the responsible development, deployment, and usage of generative models.\nGenerative AI is rapidly growing, and the market for generative AI is projected to experience significant growth in the coming years. This presents exciting business opportunities, but it also calls for continued research, innovation, and responsible practices to ensure the technology's positive impact and adherence to ethical standards.\n\nIn conclusion, generative AI holds immense promise for transforming our lives and shaping the future. By addressing challenges, promoting responsible use, and fostering collaboration, we can harness the full potential of generative AI to drive innovation, create meaningful experiences, and contribute to the betterment of society.\n","metadata":{}},{"cell_type":"markdown","source":"# Reference\n- McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (2006). A proposal for the dartmouth summer research project on artificial intelligence, August 31, 1955. AI magazine, 27(4), 12-12.\n- Smith, C., McGuire, B., Huang, T., & Yang, G. (2006). The history of artificial intelligence. University of Washington, 27.\n- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2020). Generative adversarial networks. Communications of the ACM, 63(11), 139-144.\n- Bishop, C. M., & Nasrabadi, N. M. (2006). Pattern recognition and machine learning (Vol. 4, No. 4, p. 738). New York: Springer.\n- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.\n- Murphy, K. P. (2012). Machine learning: a probabilistic perspective. MIT Press.\n- Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114.\n- Li, H., Koto, F., Wu, M., Aji, A. F., & Baldwin, T. (2023). Bactrian-X: A Multilingual Replicable Instruction-Following Model with Low-Rank Adaptation. arXiv preprint arXiv:2305.15011.\n- Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2022). Self-Instruct: Aligning Language Model with Self-Generated Instructions. arXiv preprint arXiv:2212.10560.\n- Wang, T. C., Liu, M. Y., Zhu, J. Y., Tao, A., Kautz, J., & Catanzaro, B. (2018). High-resolution image synthesis and semantic manipulation with conditional gans in Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8798-8807).\n- Ping, W., Peng, K., Zhao, K., & Song, Z. (2020, November). Waveflow: A compact flow-based model for raw audio. In International Conference on Machine Learning (pp. 7706-7716). PMLR.\n- Rudolph, M., Wandt, B., & Rosenhahn, B. (2021). Same but different: Semi-supervised defect detection with normalizing flows. In Proceedings of the IEEE/CVF winter conference on applications of computer vision (pp.907-1916).\n- Kingma, D. P., & Dhariwal, P. (2018). Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31.\n- Kumar, M., Babaeizadeh, M., Erhan, D., Finn, C., Levine, S., Dinh, L., & Kingma, D. (2019). Videoflow: A conditional flow-based model for stochastic video generation. arXiv preprint arXiv:1903.01434.\n- Hawthorne, C., Jaegle, A., Cangea, C., Borgeaud, S., Nash, C., Malinowski, M., ... & Engel, J. (2022, June). General-purpose, long-context autoregressive modeling with perceiver ar. In International Conference on Machine Learning (pp. 8535-8558).\n- Bermano, A. H., Gal, R., Alaluf, Y., Mokady, R., Nitzan, Y., Tov, O., ... & Cohen‐Or D. (2022) discuss the state-of-the-art architecture, methods, and applications of StyleGAN. Their work is published in Computer Graphics Forum, Volume 41, Issue 2, pages 591-611.\n- Manu, D., Yao, J., Liu, W., & Sun, X. (2023). GraphGANFed: A Federated Generative Framework for Graph-Structured Molecules Towards Efficient Drug Discovery. arXiv preprint arXiv:2304.05498.\n- Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature machine intelligence, 1(9), 389-399.\n- Törnberg, P. (2023). Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. arXiv preprint arXiv:2304.06588.\n- OpenAI. (2021). ChatGPT [Computer software]. Retrieved from https://openai.com\n\n","metadata":{}},{"cell_type":"markdown","source":"# Submission File","metadata":{}},{"cell_type":"code","source":"import numpy as np \nimport pandas as pd\n\nsubmission=pd.read_csv(\"/kaggle/input/2023-kaggle-ai-report/sample_submission.csv\")\nsubmission.head()","metadata":{"execution":{"iopub.status.busy":"2023-07-05T18:31:19.602189Z","iopub.execute_input":"2023-07-05T18:31:19.602688Z","iopub.status.idle":"2023-07-05T18:31:21.712402Z","shell.execute_reply.started":"2023-07-05T18:31:19.602646Z","shell.execute_reply":"2023-07-05T18:31:21.711308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.loc[0]['value']='Generative AI'\nsubmission.loc[1]['value']='https://www.kaggle.com/code/sumanadhikari04/unveiling-the-evolution-of-generativeai'\nsubmission.loc[2]['value']='https://www.kaggle.com/code/trushk/2023-kaggle-ai-report-generative-ai/comments#2331757'\nsubmission.loc[3]['value']='https://www.kaggle.com/code/muhammadirfanakbar/data-centric-ai-the-underdog-game-changer-in-ai/comments#2331779'\nsubmission.loc[4]['value']='https://www.kaggle.com/code/gvyshnya/cybersecurity-ai-in-2022-2023/comments#2331799'\nsubmission.head()","metadata":{"execution":{"iopub.status.busy":"2023-07-05T18:36:08.946072Z","iopub.execute_input":"2023-07-05T18:36:08.947370Z","iopub.status.idle":"2023-07-05T18:36:08.965513Z","shell.execute_reply.started":"2023-07-05T18:36:08.947316Z","shell.execute_reply":"2023-07-05T18:36:08.963583Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv('submission.csv',index=False)","metadata":{"execution":{"iopub.status.busy":"2023-07-05T18:38:39.025441Z","iopub.execute_input":"2023-07-05T18:38:39.025868Z","iopub.status.idle":"2023-07-05T18:38:39.034836Z","shell.execute_reply.started":"2023-07-05T18:38:39.025838Z","shell.execute_reply":"2023-07-05T18:38:39.032862Z"},"trusted":true},"execution_count":null,"outputs":[]}]}