{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Generative AI: A Panoramic Discourse \n***","metadata":{}},{"cell_type":"markdown","source":"## 1.0 Introduction\n\nIt is 2023. We are currently riding a wave of AI euphoria. Every day, Generative AI technologies are redefining the art of the possible. AI, and in particular Generative AI, continues to spew wonders, whereby machine created content parallels the crests and the quests of human creativity. Nations around the globe, are scrambling to catch up with AI innovation, so that they can rein it, regulate it and dictate what is permissible, before it is too late for unintended, inadvertent harm to take effect. Meanwhile, AI startups around the world are raising millions of dollars in venture capital funding, with promises to improve upon SOTA (state-of-the-art) Generative AI capabilities and with an eye towards disrupting the status quo – be it for artists of all stripes, or at the workplace or for knowledge intensive industries.\n\nThis essay begins with exploring the hierarchical lineage of Generative AI followed by a discussion on the four waves of innovation in Generative AI thus far. Given the resounding success of Large Language Models (LLMs) in 2023, five strategies for harnessing the power of LLMs are discussed. Finally, we discuss how current Generative AI capabilities measure against the standard of becoming completely autonomous and the caveats that call for a modicum of restraint on the part of AI enthusiasts.\n\n## 2.0 Artificial Intelligence - a brief background \n\nArtificial Intelligence is the discipline of creating, intelligent, autonomous agents that can perform tasks, with the intended fitness for use and without having to explicitly program them to accomplish these tasks. AI originated in the 1950s and for the first few decades, was under the purview of the Jedi of computer science. The era of “AI Winter” [1] followed fairly quickly and lasted until the early 1990s. “AI Winter” was a time when interest and funding in AI projects waned considerably as AI developed a reputation to overpromise and underdeliver. Simpler forms of AI, such as rule-based systems, engineered to catalog the knowledge of human experts, saw more traction, and led to the emergence of autonomous machine agents that functioned as domain experts. The emphasis was on achieving functional competence based on logic and symbolic reasoning instead of simulating biological wiring to accomplish tasks.\n\n\nFig 1: History of AI \nSource:https://www.actuaries.digital/2018/09/05/history-of-ai-winters/\n
\n\nBecause of AI winter, Machine Learning (ML), technically a sub-field of AI, began to be treated largely as an independent field. Starting with the mid-1980s, with the introduction of decision trees, and the subsequent introduction of boosting algorithms [2] and support-vector machines in the 1990s, ML began to evolve rapidly. With the advent of the Internet and the unprecedented growth of digital data, data mining, a process for analyzing large datasets for patterns and extracting insights that either improved the bottom line or pushed the top line or both, became the new normal for business houses around the world.\n\n\nAt the turn of the 21st century, Deep Learning, a sub-field of ML, which had been languishing behind in terms of technological advances by far, was tapped to work with data that is in the form of images or audio, which the traditional ML algorithms of the time, could not wrangle efficiently. Twenty years after Geoffrey Hinton introduced the concept of backpropagation in neural networks [3] to the world, in 2006, he introduced yet another groundbreaking concept of utilizing greedy learning to configure the parameters of the neural network at the time of training [4]. Between 2006 and 2009, the field of Deep Learning evolved rapidly, and after a long hiatus, neural networks, the cornerstone of AI, were back in business. Several deep learning applications flooded the market including Alexa by Amazon, Siri for smartphones, Face Id on smartphones, self-driving Smart Cars, to name a few.\n\n\n## 3.0 The era of Generative AI \n\nGenerative AI is a sub-field of Deep Learning, that employs AI techniques to generate new content. The content can be in the form of text, images, videos, music, speech and even code. Broadly speaking, since its inception, there have been four waves of breakthroughs in Generative AI - Image Generation Technologies, Transformers, Large Language Models (LLMs) and Multimodal systems.\n\n","metadata":{}},{"cell_type":"markdown","source":"\nFig 2: The relationship between AI, ML, Deep Learning and Generative AI\nSource: Created by the author\n","metadata":{}},{"cell_type":"markdown","source":"### 3.1 Image Generation \n\n VAEs - The earliest avatars of Generative AI, relied heavily on statistical models, to generate new content. Variational Autoencoders (VAEs), that could encode input images to a latent space, followed by confounding the latent space to decode new images, used a combination of Bayesian machine learning and deep learning techniques to facilitate the objective of generating new images. \n\n GANs - On the heels of VAEs, in 2014, Ian Goodfellow and his colleagues devised Generator Adversarial Networks (GANs) [5]. In its simplest form, GANs, comprise of two neural networks (or models). One model is known as the generator and the other model is known as the discriminator and these two models are analogous to the real-world actors of the art forger, and the art inspector respectively. The generator model or the art forger, generates fake images. The discriminator model or the art inspector evaluates the images created by the generator and assigns a score that lets the generator know how realistic its fake image is, thereby triggering the generator to improve upon the process of image generation, until it has successfully convinced the discriminator into thinking that it has created a realistic image. An ongoing, “Getting Started” competition on Kaggle [6] is to generate images in the style of Monet. The objective can be acheived using CycleGANs, a variant of the original GAN, that facilitates style transfer between two domains, allowing end users to transfer Monet's style of painting to contemporary photographs.\n\n Diffusion Models - In 2015, Sohl-Dickstein et. al [7], proposed the Diffusion model, that could learn the latent structure of images and use this knowledge to generate new images. The Diffusion model comprises of two processes - the forward diffusion process and the reverse diffusion process. In forward diffusion, noise is added iteratively to images and in reverse diffusion the model removes predicted noise from the image, thereby learning from the difference between the noise that was added and the noise it flagged for removal. Such a model, trained to identify noise in an image, is now capable of taking any image as input and will denoise it, thereby generating, new images. Even though Diffusion models were introduced in 2015, it was only 2020 and onwards, that this technology was actively harnessed for the purposes of image generation.\n\nFig 3 below, shows the Generative Learning Trilemma, a term coined by Xiao et. al [8], that demonstrates the strengths and weaknesses of the mainstream image generation models, relative to each other. As an example, Datatonic, a UK-based, leading AI Consultancy firm, created the Fashion Generator[9] application, using VAEs instead of GANs and Diffusion models, given the diversity afforded by VAEs. \n\n","metadata":{}},{"cell_type":"markdown","source":"\nFig 3: Generative Learning Trilemma \nCollated by the author from the below sources
\nSource 1: https://data-science-blog.com/blog/2022/05/20/generative-adversarial-networks/
\nSource 2: https://arxiv.org/pdf/2112.07804.pdf
\nSource 3: https://arxiv.org/abs/2302.07730
\nSource 4: https://data-science-blog.com/blog/2022/04/19/variational-autoencoders/
\n","metadata":{}},{"cell_type":"markdown","source":" The enduring power of Image Generation technologies \n\nAlthough the architectures discussed above were introduced to the ML community in 2014 and 2015, all of these innovations, are progressive technologies that continue to retain both currency and relevancy on the Generative AI landscape. \n\nProGANs (Progressive GANs), which facilitate the generation of higher resolution images, StyleGANs, which diffuse style elements in the image synthesis process, are variants of GANs that extend the capabilities of the original GAN framework. In March 2023, Skorokhodov et. al [10], introduced the 3DGP (3D Generator with Generic Priors), a 3D GAN that was integrated with a depth adapter and a camera model, to generate high fidelity, 3D representations of images, from a relatively under curated dataset, ImageNet, which contains 2D images. In fact, given the number of advances, made since the inception of GANs, Kang et. al [11] have created an evaluation framework for GANs, called StudioGAN. Diffusion models power contemporary image generators such as OpenAI's DALL-E and Google's Imagen. \n\n\n### 3.2 Transformers \n\nThe next wave in the field of Generative AI came in 2017, with the introduction of the Transformer architecture that was disseminated in the seminal paper, “Attention Is All You Need” [12]. Transformers use a mechanism called attention that learns the contextual relationships between words in text. Since its inception, several Transformer models have emerged, as can be seen in Fig 3. According to [13], the top five transformer models for 2023, are BERT, GPT-3, RoBERTa, T5 and ELECTRA.\n\n\nFig 4: Transformer Models \nSource:https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers_chrono.svg ","metadata":{}},{"cell_type":"markdown","source":"With the advent of Transformers, a new trend started to emerge, for acheiving generalizability in Machine Learning. The traditional ML approach, is to train models, for specific tasks, over specific training datasets, which makes these models narrow experts and that too for a limited period of time, given data drift. Generalizability of ML models, which is the degree to which models can adapt to new and unforeseen data, has continued to haunt the ML community, especially in contexts of harnessing these models in industrial settings.\n\nA counter-intuitive approach to achieving generalizability of models, has been the introduction of pre-trained models in recent years. Instead of training a model with a specific dataset for a specific task, as is done in traditional ML, pre-trained models are neural networks, the parameters, and weights of which have been configured vis-à-vis training over large corpus of data sets. Pre-trained models have shown to outperform models that are trained on specific datasets to perform specific tasks.\n\n\nFig 5: The Classic Showdown \nSource:https://medium.com/@wuyalixq/random-thoughts-about-specialists-generalists-and-the-emergence-of-chatgpt-7cd124ad6584 \n\n
\n\nOpenAI's Alec Radford and team [14] demonstrated that language models, subject to unsupervised learning, on large sets of diverse data from various domains, not only learned multiple tasks, such as reading comprehension, summarization, translation but also outperformed models that were trained to perform one of these specific tasks with specific training data.\n\nAkin to the genesis of image generation, the science of text generation, was initially rooted in statistical models that used probability distributions of words in sentences to perform NLP tasks. However, modern language models, are Large Language Models, also known as LLMs, that leverage the Transformer architecture coupled with deep learning techniques to train on large corpus of datasets that comprise of natural language. LLMs can perform both Generative AI tasks such as text summarization, text generation, question answering, sentiment analysis as well as tasks that do not fall in the realm of Generative AI such as text classification, topic selection and embeddings. GPT (Generative Pre-Trained Transformer) is the result of training on massive amounts of data and the later iterations of GPT comprise of billions of parameters.","metadata":{}},{"cell_type":"markdown","source":"\n Table 1: GPT models by size\n \n \n | GPT Model | \n Debut Date | \n # of parameters | \n
\n \n \n \n | GPT-1 | \n June 2018 | \n 117 M | \n
\n \n | GPT-2 | \n Feb 2019 | \n 1.5 B | \n
\n \n | GPT-3 | \n June 2020 | \n 175 B | \n
\n \n | GPT-4 | \n March 2023 | \n Unknown | \n
\n \n
\n\n","metadata":{}},{"cell_type":"markdown","source":"The consensus in the industry is that \"bigger is better\". The larger the model, i.e., the size of the model in terms of number of parameters, the more capable it is at performing tasks. Currently only the behemoths of the tech industry, such as Google, Facebook, Microsoft and well-funded startups have the wherewithal to undertake the development of industry standard LLMs.\n\nChen et al., in their paper titled, “Evaluating Large Language Models Trained on Code” [15], released Codex, that could generate Python code in response to docstrings. Codex was borne out of fine-tuning GPT models on publicly available code in GitHub with the objective of generating standalone Python functions. Today, we have several AI powered code assistants, that generate code based on prompts and nudge the developer with suggested code snippets. GitHub Copilot, tops the list of the best AI coding assistants in the market in 2023 [16].\n\nIn order to acheive optimal performance, pre-trained models, such as GPT, are typically fine-tuned for specific domains or for specific tasks. An example of this is BloombergGPT, that was created by training the GPT model on enormous amounts of financial data in order to perform NLP tasks in the financial domain. What was once the classic showdown between the specialists and the generalists has turned into a melding of the minds with the specialists standing tall on the shoulders of generalists.\n\nTransformers have been identified as foundation models for AI [17]. Foundation models are models that are based on deep neural networks and self-supervised learning and are centrally critical to the development of the field of AI. Foundation models inherently embody the properties of emergence, i.e. the ability to scale to new capabilities and incentivize homogenization, i.e. consolidation of methodologies for building AI applications in a variety of domains.\n\n### 3.3 Large Language models (LLMs) \n\nLLMs are all the rage in 2023. There are two types of LLMs, proprietary LLMs and open-source LLMs. Proprietary LLMs are owned by an organization and are only available to customers who purchase a license to use it. The top rated LLMs for 2023 [18] are proprietary LLMs and leading the charge is OpenAI. Open-source LLMs are developed as public, collaborative initiatives and are freely available for public use and consumption. HuggingFace is a community for democratizing AI, that pushes back, against the tenets of closed, proprietary AI models and makes available, open-source models that can be downloaded and used freely for developing AI applications. HuggingFace categorizes the models by the tasks that they are capable of performing. For example, for the task of text summarization, several models are available, including variants of BART, Pegasus, T5, specifically tuned for the task of text summarization. A comprehensive list of open-source LLMs can be found at this link [19].\n\n#### 3.3.1 Five strategies for unlocking the value of LLMs\n\nGiven the sudden upsurge in the number of LLMs in the market, several initiatives have emerged to capitalize on the prowess of LLMs. There are various strategies for harnessing the value of LLMs.\n\n#### i. Count your tokens before they are embedded\n\nTextual, unstructured data, when fed to LLMs, is mapped to tokens using specific encoders. TikToken, is an open-source Python package that enables users to determine the number of tokens that would be consumed by OpenAI models. Why is this important? Using the capabilities of proprietary LLMs comes at a price and the cost is directly proportional to the number of API calls made to the model and the number of tokens used during each API call. Also, each LLM has an upper bound on the number of tokens the LLM can leverage for both processing the input text and parsing the output text, also known as the context window, hence it is a good idea to estimate the number of tokens that will be needed to accomplish the task of processing the entire corpus of text at hand.","metadata":{}},{"cell_type":"markdown","source":"\n Table 2: Context window for LLMs \n \n \n | LLM Provider | \n LLM | \n Type | \n No. of tokens | \n
\n \n \n \n | OpenAI | \n text-davinci-003 | \n Proprietary | \n 4097 | \n
\n \n | OpenAI | \n gpt-3.5-turbo | \n Proprietary | \n 4097 | \n
\n \n | Anthrope | \n Claude | \n Proprietary | \n 100k | \n
\n \n | HuggingFace | \n BERT | \n Open-source | \n 512 | \n
\n \n | HuggingFace | \n BART | \n Open-source | \n 1024 | \n
\n \n | HuggingFace | \n Longformer | \n Open-source | \n 16k | \n
\n \n
","metadata":{}},{"cell_type":"markdown","source":"#### ii. Prompt Engineering\n\nThe input provided to the LLM is known as the prompt and the quality of LLM’s response is highly contingent on the prompt provided to it. “Prompt engineering” focuses on consciously creating prompts, for obtaining high-quality output from LLMs. Patterns of prompts have been devised that can increase the probability of getting desired responses from the LLMs. Zero-shot prompts, one-shot prompts, chain-of-thought prompts, role-based prompts are various strategies that focus on optimizing the input to the LLM to get the desired response.\n\n#### iii. Fine-tuning LLMs\n\nBuilding an LLM from scratch is a resource-intensive initiative, and is currently pursued by the likes of tech giants. Fine tuning refers to re-training an existing LLM with a specific dataset in a specific domain. Fine-tuning an LLM results in modifying the weights and parameters of that LLM and sharpens the LLM's ability to generate responses that are context aware as well as enhances the usability of LLMs for a specific domain or task. CaseText is a firm in the Legal space, that has fine-tuned GPT-4 to create CoCounsel, that assists with legal research, relevant case law retrieval and creates drafts of depositions, in a matter of minutes. CaseText was recently acquired by ThomasReuters, largely on account of its unprecedented, cutting-edge capabilities in legal research technology.\n\n#### iv. Retrieval Augmented Generation (RAG)\n\nIn Retrieval Augmented Generation, Generative AI is used in conjunction with information-retrieval mechanisms to generate the response to the prompt. In this framework, when the end user provides the input, the input is used to first retrieve a set of relevant documents from a corpus of knowledge (non-parametric memory), and these set of documents subsequently serve as the context for the input. Both the input, and the context are then provided to the LLM (parametric memory) to generate the response. This framework enables the user to anchor the original input with contextual information that cues the LLM on the expected response. RAG is a technique for cluing in the LLM on the desired response, without having to retrain the LLM.\n\n#### v. LangChain\n\nLangChain [20], developed by Harrison Chase, is a powerful framework for building LLM centric applications. LangChain does not purport its own LLM, but is designed to work with commonly available LLMs that are made available by other model providers. LangChain offers several abstractions that enables the end user to seamlessly interface with LLM, provide prompts using prompt templates and parse and store the output generated by LLMs.\n\nGiven the proliferation of LLMs in 2023, choosing the right LLM is a non-trivial task and will depend on several factors such as the use cases in scope for the LLM application, the cost of making API calls, the size of input text, the reliability of the model and the model vendor and most important of all the quality of output generated by the LLM.\n\n### 3.4 Multimodal models\n\nMultimodal models are deep learning models that can process one or more data types, such as images, audio and text, simultaneously and can generate content that is different from the data type(s) input to the model either in terms of modality or in terms of cardinality. For example, OpenAI's GPT-4 is a multi-modal model that can take as input, both text and image and in response, generate textual output. \n\nModern text-to-image generative AI applications such as StableDiffusion, DALL-E2 use Diffusion models in conjunction with LLMs to generate realistic images. OpenAI's DALL-E now offers APIs [21] that can take in text prompts and send back images that are generated in response to the prompt. The below image was generated using MultiViewDiffusion [22], a text to panoramic image generator. The prompt provided was \"This tree house located deep inside an Amazonian jungle blend both modern and rustic designs.\"","metadata":{}},{"cell_type":"markdown","source":"\nFig 6: Image generated by a Text to Panoramic image generator\nSource: Image generated by the author using MultiViewDiffusion","metadata":{}},{"cell_type":"markdown","source":"Another interesting example of a multimodal model, is MusicLM, released by Google in 2023. MusicLM can take in a textual prompt and in response to the prompt generate music at 24 kHz that plays for a few minutes [23]. It appears that the jury is still out on the top text-to-video AI generators to consider in 2023. I reviewed three different listicles [24], [25], [26], on the top text-to-AI video software available in the market today and all three listicles had their own version of the best AI video generators to use. Only InVideo and Synthesia were common across all three lists. InVideo takes in a movie script and converts it to a video. Synthesia, facilitates the creation of professional videos that can be used in work environments and is touted for its range of digital avatars.\n\nA major area of concern, for multimodal models in particular, has been ethics. DALL-E allows users to upload images for editing, and there are no guardrails in place that would prevent a user from uploading an image for which the user does not have a copyright, other than a reminder to the user, at the time of image upload, to take a moment to consider the content that the user is uploading to the application. Data privacy concerns are heightened when multimodal data is included, as data could include facial features, copyrighted content, and the model may make unexpected inferences that could reveal personal information and mimic toxic biases.","metadata":{}},{"cell_type":"markdown","source":"## 4.0 Quality Control for Generative AI applications\n***\n \"Quality is never an accident. It is always the result of intelligent efforts.\" \n - John Ruskin
\n\n***\n\nWhile state-of-art Generative AI applications can generate content in various formats, their Achilles heel is the absence of a feedback loop that tells the applications what they are doing right and where they need to improve. Such a mechanism is needed to make Generative AI applications truly autonomous. Currently the onus is on the end user to tune their inputs so that the application generates a desired response and there can be a number of iterations before the application gets it right. Hence, as they exist today, Generative AI applications are touted as sounding boards for ideas. \n\nCurrently, one of the best practices of the governance frameworks for LLMs deployed within organizational contexts, is to have human feedback to control for factors such as AI hallucination, chaffing out incorrect responses, tailoring a tirade of content. It has become a given that LLMs, like humans, have limitations and are prone to error, but something is better than nothing.\n\nKaggle can facilitate complementary capabilities to the advances made in the field of Generative AI, by hosting problem statements that focus on the discriminative abilities of AI. A review of Kaggle competitions held in the past few years identified that some of these competitions were designed to acheive this objective, such as:\n\n\n - Feedback Prize - English Language Learning
\n - Feedback Prize - Evaluating Student Writing
\n - Feedback Prize - Predicting Effective Arguments
\n
\n\nAll of the above competitions had the objective of developing algorithms that could facilitate automated writing assessments, thereby providing a degree of feedback on the quality of writing produced, albeit by humans. The 1st place solutions, for all the above competitions [27], ensembled multiple transformer models, that were typically variants of BERT, to arrive at the winning solution. \n\nStandards Development Organizations (SDOs) should be incentivized to develop standards to ensure the development, deployment and usage of Generative AI applications that are effective, safe and unbiased. Methods such as inspections, reviews, audits as well as metrics that measure the degree of bias, are needed, to quantify the extent to which Generative AI applications are saturated with the three categories of AI bias [28] - statistical and computational bias, human bias and institutional bias and should accordingly trigger, implementation of bias mitigation plans, which can include notification, reconstitution and even dismantlement of highly biased applications. \n\nRegulations, enacted by nations, is another mechanism for installing controls on the content that is generated vis-a-vis AI tools. Generative AI, if allowed to run wild and unchecked, is capable of perpetuating bias, spreading misinformation and violating privacy rules. In June 2023, the European Parliament passed the European Union Artificial Intelligence Act [29], that is regulation designed to identify and classify high-risk AI applications that can be harmful if left unregulated. The regulation aims to strike a balance between harnessing the power of AI for societal benefit and mitigating the risks associated with the use of the technology with the \"ultimate aim of increasing human well-being.\"\n\n## 5.0 Conclusion\n\nAI is no longer restricted to the strata of rarefied minds as was the case at the time of its inception. Plug and play transformers, readily available on the HuggingFace community, can be used to build savvy Generative AI applications in a matter of minutes. \n\nChatGPT released in Nov 2022, has induced imaginations around the world on the art of the possible. On the heels of ChatGPT, in May 2023, Google launched Search Labs that provides end users with AI-powered experiences. I was granted access to the Beta version of Search Labs. When I typed in a search term in Google’s search bar, a Generative AI response, augmented on top of the regular search results was presented. The winning feature is a set of suggested prompts, that are displayed at the bottom of the Generative AI response, that the end user can click to continue the user’s quest for information.\n\nEasily accessible, AI is now set to become as commonplace as the Internet. Just like Internet, made it possible, for every individual to have their own personal email Id, Generative AI, will make it possible for everyone to have an AI-wallet stacked with digital assistants that will enable the individual to achieve their goals, both personally and professionally. ","metadata":{}},{"cell_type":"markdown","source":"\n# References\n\n[1] https://www.actuaries.digital/2018/09/05/history-of-ai-winters/\n\n[2] R.E. Schapire, The Strength of Weak Learnability, Machine Learning 5, (1990): 197–227 https://doi.org/10.1007/BF00116037\n\n[3] D.E. Rummelahrt, G.E. Hinton and R.J. Williams, “Learning Representations by Back-Propagating Errors,” Nature 323, no. 6088 (1986), 533 - 536\n\n[4] G.E. Hinton, S. Osindero and Y. Teh, “A Fast Learning Algorithm for Deep Belief Nets,” Neural Computation 18, no. 7 (2006): 1527 - 1554\n\n[5] I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, “Generative Adversarial Nets” Advances in Neural Information Processing Systems, 2014. https://arxiv.org/pdf/1406.2661.pdf\n\n[6] https://www.kaggle.com/competitions/gan-getting-started\n\n[7] J. Sohl-Dickstein, E.A. Weiss, N. Maheshwaranathan, S. Ganguli, “Deep Unsupervised Learning using Nonequilibrium Thermodynamics” 2015, https://doi.org/10.48550/arXiv.1503.03585 \n\n[8] Z. Xiao, K. Kreis, A. Vahdat, \"Tackling the Generative Learning Trilemma with Denoising Diffusion GANs\", 2021, https://arxiv.org/abs/2112.07804\n\n[9] https://datatonic.com/insights/computer-vision-generative-models-conditional-image-synthesis/\n\n[10] https://snap-research.github.io/3dgp/\n\n[11] M. Kang, J. Shin, J. Park, \"StudioGAN: A Taxonomy and Benchmark\nof GANs for Image Synthesis\", https://arxiv.org/pdf/2206.09479v2.pdf, 2022\n\n[12] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, I. Polosukhin, \"Attention is All You Need\", Advances in Neural Information Processing Systems 30 (NIPS 2017), https://doi.org/10.48550/arXiv.1706.03762\n\n[13] https://www.codementor.io/@alinakhay/5-top-most-powerful-transformer-models-2023-24hb9azuzn\n\n[14] A Radford, J Wu, R Child, D Luan, D Amodei, I Sutskever, \"Language Models are Unspuervised Multitask Learners\",(2019), https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf\n\n[15] Chen et.al, \"Evaluating Large Language Models Trained on Code\", 2021, \nhttps://doi.org/10.48550/arXiv.2107.03374\n\n[16] https://www.elegantthemes.com/blog/wordpress/best-ai-coding-assistant#:~:text=Which%20is%20the%20best%20AI,is%20powered%20by%20OpenAI's%20Codex.\n\n[17] R. Bommasani et. al. “On the Opportunities and Risks of Foundation Models , https://doi.org/10.48550/arXiv.2108.07258\n\n[18] https://sourceforge.net/software/large-language-models/\n\n[19] https://medium.datadriveninvestor.com/list-of-open-source-large-language-models-llms-4eac551bda2e\n\n[20] https://python.langchain.com/docs/get_started/introduction.html\n\n[21] https://platform.openai.com/docs/guides/images/language-specific-tips\n\n[22] https://huggingface.co/spaces/tangshitao/MVDiffusion\n\n[23] Agostinelli et.al, \"MusicLM: Generating music from text\", https://doi.org/10.48550/arXiv.2301.11325\n\n[24] https://neilchasefilm.com/video-generators/\n\n[25] https://zapier.com/blog/best-ai-video-generator/\n\n[26] https://www.unite.ai/best-ai-video-generators/\n\n[27]https://www.nist.gov/publications/towards-standard-identifying-and-managing-bias-artificial-intelligence\n\n[28] /kaggle/input/2023-kaggle-ai-report/kaggle_writeups_20230510.csv\n\n[29] https://artificialintelligenceact.eu/the-act/\n\n[30] T. J. Sejnowski. (2018). The Deep Learning Revolution. MIT Press.\n\n***\n","metadata":{}},{"cell_type":"markdown","source":"## Submission","metadata":{}},{"cell_type":"code","source":"import pandas as pd\n\nsubmission = pd.read_csv(\"/kaggle/input/2023-kaggle-ai-report/sample_submission.csv\")\nsubmission.loc[0]['value'] = 'Generative AI'\nsubmission.loc[1]['value'] = 'https://www.kaggle.com/code/gouriprakash/generative-ai-a-panoramic-discourse'\nsubmission.loc[2]['value'] = 'https://www.kaggle.com/code/satarupadeb/role-of-generative-ai-in-industry-5-0/comments#2333418'\nsubmission.loc[3]['value'] = 'https://www.kaggle.com/code/dgoldberg98/team-rangers/comments#2339596'\nsubmission.loc[4]['value'] = 'https://www.kaggle.com/code/chinmaypagey/generative-ai-report-by-team-sussy-baka/comments#2342445'\n\nsubmission.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2023-07-16T20:49:09.711496Z","iopub.execute_input":"2023-07-16T20:49:09.712334Z","iopub.status.idle":"2023-07-16T20:49:09.729565Z","shell.execute_reply.started":"2023-07-16T20:49:09.712294Z","shell.execute_reply":"2023-07-16T20:49:09.728322Z"},"trusted":true},"execution_count":null,"outputs":[]}]}![\"Heat](\"https://www.actuaries.digital/wp-content/uploads/2018/08/ai1.jpg\")
![\"Heat](\"https://gouriprakash.com/wp-content/uploads/2023/07/Image4.jpg\")
![\"Heat](\"https://gouriprakash.com/wp-content/uploads/2023/07/GLT-1024x532.jpg\")
![\"Heat](\"https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers_chrono.svg\")
![\"Heat](\"https://miro.medium.com/v2/resize:fit:640/format:webp/1*pAo_wfIeq5RBZYt-XQAhnw.jpeg\")
![\"Heat](\"https://gouriprakash.com/wp-content/uploads/2023/07/Treehouse_JPG-1024x242.jpg\")