07. Recurrent Neural Networks/06. Bidirectional Recurrent Neural Networks - MXNet.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Bidirectional Recurrent Neural Networks - MXNet (From Scratch).ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "-5kJja-laU8k",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 151
        },
        "outputId": "9b386bee-f601-4bbf-b3ce-0e23cedd4bf1"
      },
      "source": [
        "!pip install mxnet-cu100"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: mxnet-cu100 in /usr/local/lib/python3.6/dist-packages (1.4.1)\n",
            "Requirement already satisfied: requests>=2.20.0 in /usr/local/lib/python3.6/dist-packages (from mxnet-cu100) (2.21.0)\n",
            "Requirement already satisfied: numpy<1.15.0,>=1.8.2 in /usr/local/lib/python3.6/dist-packages (from mxnet-cu100) (1.14.6)\n",
            "Requirement already satisfied: graphviz<0.9.0,>=0.8.1 in /usr/local/lib/python3.6/dist-packages (from mxnet-cu100) (0.8.4)\n",
            "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.20.0->mxnet-cu100) (2.8)\n",
            "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.20.0->mxnet-cu100) (3.0.4)\n",
            "Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests>=2.20.0->mxnet-cu100) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.20.0->mxnet-cu100) (2019.6.16)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kPljrJ7Fkxoh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import mxnet as mx\n",
        "from mxnet import nd, autograd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "ctx = mx.gpu()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DK1pMOcRlVgE",
        "colab_type": "text"
      },
      "source": [
        "##Reading the data from text file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5aBCNwVSlYpT",
        "colab_type": "code",
        "outputId": "c8ed8a9f-609d-415b-cff8-cac740acdc81",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 289
        }
      },
      "source": [
        "with open('text.txt') as file:\n",
        "    text = file.read()\n",
        "\n",
        "print(text)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Artificial intelligence (AI) is the simulation of human intelligence processes by machines, especially computer systems. These processes include learning (the acquisition of information and rules for using the information), reasoning (using rules to reach approximate or definite conclusions) and self-correction.\n",
            "Particular applications of AI include expert systems, speech recognition and machine vision. Artificial intelligence (AI), the ability of a digital computer or computer-controlled robot to perform tasks commonly associated with intelligent beings. The term is frequently applied to the project of developing systems endowed with the intellectual processes characteristic of humans, such as the ability to reason, discover meaning, generalize, or learn from past experience. Since the development of the digital computer in the 1940s, it has been demonstrated that computers can be programmed to carry out very complex tasks—as, for example, discovering proofs for mathematical theorems or playing chess—with great proficiency.\n",
            "Still, despite continuing advances in computer processing speed and memory capacity, there are as yet no programs that can match human flexibility over wider domains or in tasks requiring much everyday knowledge. On the other hand, some programs have attained the performance levels of human experts and professionals in performing certain specific tasks, so that artificial intelligence in this limited sense is found in applications as diverse as medical diagnosis, computer search engines, and voice or handwriting recognition.\n",
            "All but the simplest human behaviour is ascribed to intelligence, while even the most complicated insect behaviour is never taken as an indication of intelligence. What is the difference? Consider the behaviour of the digger wasp, Sphex ichneumoneus. When the female wasp returns to her burrow with food, she first deposits it on the threshold, checks for intruders inside her burrow, and only then, if the coast is clear, carries her food inside. The real nature of the wasp’s instinctual behaviour is revealed if the food is moved a few inches away from the entrance to her burrow while she is inside: on emerging, she will repeat the whole procedure as often as the food is displaced. Intelligence—conspicuously absent in the case of Sphex—must include the ability to adapt to new circumstances.\n",
            "Psychologists generally do not characterize human intelligence by just one trait but by the combination of many diverse abilities. Research in AI has focused chiefly on the following components of intelligence: learning, reasoning, problem solving, perception, and using language.\n",
            "AI can be categorized as either weak or strong. Weak AI, also known as narrow AI, is an AI system that is designed and trained for a particular task. Virtual personal assistants, such as Apple's Siri, are a form of weak AI. Strong AI, also known as artificial general intelligence, is an AI system with generalized human cognitive abilities. When presented with an unfamiliar task, a strong AI system is able to find a solution without human intervention.\n",
            "Because hardware, software and staffing costs for AI can be expensive, many vendors are including AI components in their standard offerings, as well as access to Artificial Intelligence as a Service (AIaaS) platforms. AI as a Service allows individuals and companies to experiment with AI for various business purposes and sample multiple platforms before making a commitment. Popular AI cloud offerings include Amazon AI services, IBM Watson Assistant, Microsoft Cognitive Services and Google AI services. While AI tools present a range of new functionality for businesses ,the use of artificial intelligence raises ethical questions. This is because deep learning algorithms, which underpin many of the most advanced AI tools, are only as smart as the data they are given in training. Because a human selects what data should be used for training an AI program, the potential for human bias is inherent and must be monitored closely.\n",
            "Some industry experts believe that the term artificial intelligence is too closely linked to popular culture, causing the general public to have unrealistic fears about artificial intelligence and improbable expectations about how it will change the workplace and life in general.\n",
            "Researchers and marketers hope the label augmented intelligence, which has a more neutral connotation, will help people understand that AI will simply improve products and services, not replace the humans that use them. Knowledge engineering is a core part of AI research. Machines can often act and react like humans only if they have abundant information relating to the world. Artificial intelligence must have access to objects, categories, properties and relations between all of them to implement knowledge engineering. Initiating common sense, reasoning and problem-solving power in machines is a difficult and tedious task.Machine learning is also a core part of AI.\n",
            "There are a number of different forms of learning as applied to artificial intelligence. The simplest is learning by trial and error. For example, a simple computer program for solving mate-in-one chess problems might try moves at random until mate is found. The program might then store the solution with the position so that the next time the computer encountered the same position it would recall the solution. This simple memorizing of individual items and procedures—known as rote learning—is relatively easy to implement on a computer. More challenging is the problem of implementing what is called generalization. Generalization involves applying past experience to analogous new situations. For example, a program that learns the past tense of regular English verbs by rote will not be able to produce the past tense of a word such as jump unless it previously had been presented with jumped, whereas a program that is able to generalize can learn the “add ed” rule and so form the past tense of jump based on experience with similar verbs.\n",
            "Learning without any kind of supervision requires an ability to identify patterns in streams of inputs, whereas learning with adequate supervision involves classification and numerical regressions. Classification determines the category an object belongs to and regression deals with obtaining a set of numerical input or output examples, thereby discovering functions enabling the generation of suitable outputs from respective inputs. Mathematical analysis of machine learning algorithms and their performance is a well-defined branch of theoretical computer science often referred to as computational learning theory. Machine perception deals with the capability to use sensory inputs to deduce the different aspects of the world, while computer vision is the power to analyze visual inputs with a few sub-problems such as facial, object and gesture recognition. Robotics is also a major field related to AI. Robots require intelligence to handle tasks such as object manipulation and navigation, along with sub-problems of localization, motion planning and mapping.\n",
            "To reason is to draw inferences appropriate to the situation. Inferences are classified as either deductive or inductive. An example of the former is, “Fred must be in either the museum or the café. He is not in the café; therefore he is in the museum,” and of the latter, “Previous accidents of this sort were caused by instrument failure; therefore this accident was caused by instrument failure.” The most significant difference between these forms of reasoning is that in the deductive case the truth of the premises guarantees the truth of the conclusion, whereas in the inductive case the truth of the premise lends support to the conclusion without giving absolute assurance. Inductive reasoning is common in science, where data are collected and tentative models are developed to describe and predict future behaviour—until the appearance of anomalous data forces the model to be revised. Deductive reasoning is common in mathematics and logic, where elaborate structures of irrefutable theorems are built up from a small set of basic axioms and rules.\n",
            "There has been considerable success in programming computers to draw inferences, especially deductive inferences. However, true reasoning involves more than just drawing inferences; it involves drawing inferences relevant to the solution of the particular task or situation. This is one of the hardest problems confronting AI.\n",
            "Problem solving, particularly in artificial intelligence, may be characterized as a systematic search through a range of possible actions in order to reach some predefined goal or solution. Problem-solving methods divide into special purpose and general purpose. A special-purpose method is tailor-made for a particular problem and often exploits very specific features of the situation in which the problem is embedded. In contrast, a general-purpose method is applicable to a wide variety of problems. One general-purpose technique used in AI is means-end analysis—a step-by-step, or incremental, reduction of the difference between the current state and the final goal. The program selects actions from a list of means—in the case of a simple robot this might consist of PICKUP, PUTDOWN, MOVEFORWARD, MOVEBACK, MOVELEFT, and MOVERIGHT—until the goal is reached.\n",
            "In perception the environment is scanned by means of various sensory organs, real or artificial, and the scene is decomposed into separate objects in various spatial relationships. Analysis is complicated by the fact that an object may appear different depending on the angle from which it is viewed, the direction and intensity of illumination in the scene, and how much the object contrasts with the surrounding field.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vJcQon60lYmX",
        "colab_type": "code",
        "outputId": "9e8d5764-e466-4d0f-cf28-b540a781909d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "characters = list(set(text))\n",
        "vocab_length = len(characters)\n",
        "print(characters)\n",
        "print(f'Number of unique characters: {vocab_length}')"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['s', 'a', 'm', 'C', 'I', 'N', '0', '“', '—', '4', 'l', 'T', ';', 'g', 'z', 'U', 'G', 'p', ':', 'x', 'A', 'c', ' ', '-', 'S', ')', 't', 'b', 'r', 'd', '9', '?', 'K', 'L', 'n', 'M', 'H', 'B', 'v', 'h', 'O', '1', 'W', \"'\", 'R', 'o', 'k', '”', 'E', 'j', 'V', ',', '.', '(', '\\n', 'F', 'i', 'é', 'P', 'u', 'q', 'w', 'f', '’', 'D', 'y', 'e']\n",
            "Number of unique characters: 67\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E2LeNuEDlYhJ",
        "colab_type": "code",
        "outputId": "d71a4437-6071-44f4-c1d3-7efbf68c55bf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "char_to_int = dict([(char, i) for i, char in enumerate(characters)])\n",
        "char_to_int"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'\\n': 54,\n",
              " ' ': 22,\n",
              " \"'\": 43,\n",
              " '(': 53,\n",
              " ')': 25,\n",
              " ',': 51,\n",
              " '-': 23,\n",
              " '.': 52,\n",
              " '0': 6,\n",
              " '1': 41,\n",
              " '4': 9,\n",
              " '9': 30,\n",
              " ':': 18,\n",
              " ';': 12,\n",
              " '?': 31,\n",
              " 'A': 20,\n",
              " 'B': 37,\n",
              " 'C': 3,\n",
              " 'D': 64,\n",
              " 'E': 48,\n",
              " 'F': 55,\n",
              " 'G': 16,\n",
              " 'H': 36,\n",
              " 'I': 4,\n",
              " 'K': 32,\n",
              " 'L': 33,\n",
              " 'M': 35,\n",
              " 'N': 5,\n",
              " 'O': 40,\n",
              " 'P': 58,\n",
              " 'R': 44,\n",
              " 'S': 24,\n",
              " 'T': 11,\n",
              " 'U': 15,\n",
              " 'V': 50,\n",
              " 'W': 42,\n",
              " 'a': 1,\n",
              " 'b': 27,\n",
              " 'c': 21,\n",
              " 'd': 29,\n",
              " 'e': 66,\n",
              " 'f': 62,\n",
              " 'g': 13,\n",
              " 'h': 39,\n",
              " 'i': 56,\n",
              " 'j': 49,\n",
              " 'k': 46,\n",
              " 'l': 10,\n",
              " 'm': 2,\n",
              " 'n': 34,\n",
              " 'o': 45,\n",
              " 'p': 17,\n",
              " 'q': 60,\n",
              " 'r': 28,\n",
              " 's': 0,\n",
              " 't': 26,\n",
              " 'u': 59,\n",
              " 'v': 38,\n",
              " 'w': 61,\n",
              " 'x': 19,\n",
              " 'y': 65,\n",
              " 'z': 14,\n",
              " 'é': 57,\n",
              " '—': 8,\n",
              " '’': 63,\n",
              " '“': 7,\n",
              " '”': 47}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3A--N_D3lYe6",
        "colab_type": "code",
        "outputId": "da3d98ee-8953-41fa-f7a5-66fde976bf8a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "int_to_char = dict([(i, char) for i, char in enumerate(characters)])\n",
        "int_to_char"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{0: 's',\n",
              " 1: 'a',\n",
              " 2: 'm',\n",
              " 3: 'C',\n",
              " 4: 'I',\n",
              " 5: 'N',\n",
              " 6: '0',\n",
              " 7: '“',\n",
              " 8: '—',\n",
              " 9: '4',\n",
              " 10: 'l',\n",
              " 11: 'T',\n",
              " 12: ';',\n",
              " 13: 'g',\n",
              " 14: 'z',\n",
              " 15: 'U',\n",
              " 16: 'G',\n",
              " 17: 'p',\n",
              " 18: ':',\n",
              " 19: 'x',\n",
              " 20: 'A',\n",
              " 21: 'c',\n",
              " 22: ' ',\n",
              " 23: '-',\n",
              " 24: 'S',\n",
              " 25: ')',\n",
              " 26: 't',\n",
              " 27: 'b',\n",
              " 28: 'r',\n",
              " 29: 'd',\n",
              " 30: '9',\n",
              " 31: '?',\n",
              " 32: 'K',\n",
              " 33: 'L',\n",
              " 34: 'n',\n",
              " 35: 'M',\n",
              " 36: 'H',\n",
              " 37: 'B',\n",
              " 38: 'v',\n",
              " 39: 'h',\n",
              " 40: 'O',\n",
              " 41: '1',\n",
              " 42: 'W',\n",
              " 43: \"'\",\n",
              " 44: 'R',\n",
              " 45: 'o',\n",
              " 46: 'k',\n",
              " 47: '”',\n",
              " 48: 'E',\n",
              " 49: 'j',\n",
              " 50: 'V',\n",
              " 51: ',',\n",
              " 52: '.',\n",
              " 53: '(',\n",
              " 54: '\\n',\n",
              " 55: 'F',\n",
              " 56: 'i',\n",
              " 57: 'é',\n",
              " 58: 'P',\n",
              " 59: 'u',\n",
              " 60: 'q',\n",
              " 61: 'w',\n",
              " 62: 'f',\n",
              " 63: '’',\n",
              " 64: 'D',\n",
              " 65: 'y',\n",
              " 66: 'e'}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "58oqFgyHnkt2",
        "colab_type": "code",
        "outputId": "43800bc9-fe50-4c03-d8ba-2573defb9ef5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "int_text = [char_to_int[char] for char in text]\n",
        "int_text"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[20,\n",
              " 28,\n",
              " 26,\n",
              " 56,\n",
              " 62,\n",
              " 56,\n",
              " 21,\n",
              " 56,\n",
              " 1,\n",
              " 10,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 26,\n",
              " 66,\n",
              " 10,\n",
              " 10,\n",
              " 56,\n",
              " 13,\n",
              " 66,\n",
              " 34,\n",
              " 21,\n",
              " 66,\n",
              " 22,\n",
              " 53,\n",
              " 20,\n",
              " 4,\n",
              " 25,\n",
              " 22,\n",
              " 56,\n",
              " 0,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 0,\n",
              " 56,\n",
              " 2,\n",
              " 59,\n",
              " 10,\n",
              " 1,\n",
              " 26,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 22,\n",
              " 45,\n",
              " 62,\n",
              " 22,\n",
              " 39,\n",
              " 59,\n",
              " 2,\n",
              " 1,\n",
              " 34,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 26,\n",
              " 66,\n",
              " 10,\n",
              " 10,\n",
              " 56,\n",
              " 13,\n",
              " 66,\n",
              " 34,\n",
              " 21,\n",
              " 66,\n",
              " 22,\n",
              " 17,\n",
              " 28,\n",
              " 45,\n",
              " 21,\n",
              " 66,\n",
              " 0,\n",
              " 0,\n",
              " 66,\n",
              " 0,\n",
              " 22,\n",
              " 27,\n",
              " 65,\n",
              " 22,\n",
              " 2,\n",
              " 1,\n",
              " 21,\n",
              " 39,\n",
              " 56,\n",
              " 34,\n",
              " 66,\n",
              " 0,\n",
              " 51,\n",
              " 22,\n",
              " 66,\n",
              " 0,\n",
              " 17,\n",
              " 66,\n",
              " 21,\n",
              " 56,\n",
              " 1,\n",
              " 10,\n",
              " 10,\n",
              " 65,\n",
              " 22,\n",
              " 21,\n",
              " 45,\n",
              " 2,\n",
              " 17,\n",
              " 59,\n",
              " 26,\n",
              " 66,\n",
              " 28,\n",
              " 22,\n",
              " 0,\n",
              " 65,\n",
              " 0,\n",
              " 26,\n",
              " 66,\n",
              " 2,\n",
              " 0,\n",
              " 52,\n",
              " 22,\n",
              " 11,\n",
              " 39,\n",
              " 66,\n",
              " 0,\n",
              " 66,\n",
              " 22,\n",
              " 17,\n",
              " 28,\n",
              " 45,\n",
              " 21,\n",
              " 66,\n",
              " 0,\n",
              " 0,\n",
              " 66,\n",
              " 0,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 21,\n",
              " 10,\n",
              " 59,\n",
              " 29,\n",
              " 66,\n",
              " 22,\n",
              " 10,\n",
              " 66,\n",
              " 1,\n",
              " 28,\n",
              " 34,\n",
              " 56,\n",
              " 34,\n",
              " 13,\n",
              " 22,\n",
              " 53,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 1,\n",
              " 21,\n",
              " 60,\n",
              " 59,\n",
              " 56,\n",
              " 0,\n",
              " 56,\n",
              " 26,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 22,\n",
              " 45,\n",
              " 62,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 62,\n",
              " 45,\n",
              " 28,\n",
              " 2,\n",
              " 1,\n",
              " 26,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 22,\n",
              " 1,\n",
              " 34,\n",
              " 29,\n",
              " 22,\n",
              " 28,\n",
              " 59,\n",
              " 10,\n",
              " 66,\n",
              " 0,\n",
              " 22,\n",
              " 62,\n",
              " 45,\n",
              " 28,\n",
              " 22,\n",
              " 59,\n",
              " 0,\n",
              " 56,\n",
              " 34,\n",
              " 13,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 62,\n",
              " 45,\n",
              " 28,\n",
              " 2,\n",
              " 1,\n",
              " 26,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 25,\n",
              " 51,\n",
              " 22,\n",
              " 28,\n",
              " 66,\n",
              " 1,\n",
              " 0,\n",
              " 45,\n",
              " 34,\n",
              " 56,\n",
              " 34,\n",
              " 13,\n",
              " 22,\n",
              " 53,\n",
              " 59,\n",
              " 0,\n",
              " 56,\n",
              " 34,\n",
              " 13,\n",
              " 22,\n",
              " 28,\n",
              " 59,\n",
              " 10,\n",
              " 66,\n",
              " 0,\n",
              " 22,\n",
              " 26,\n",
              " 45,\n",
              " 22,\n",
              " 28,\n",
              " 66,\n",
              " 1,\n",
              " 21,\n",
              " 39,\n",
              " 22,\n",
              " 1,\n",
              " 17,\n",
              " 17,\n",
              " 28,\n",
              " 45,\n",
              " 19,\n",
              " 56,\n",
              " 2,\n",
              " 1,\n",
              " 26,\n",
              " 66,\n",
              " 22,\n",
              " 45,\n",
              " 28,\n",
              " 22,\n",
              " 29,\n",
              " 66,\n",
              " 62,\n",
              " 56,\n",
              " 34,\n",
              " 56,\n",
              " 26,\n",
              " 66,\n",
              " 22,\n",
              " 21,\n",
              " 45,\n",
              " 34,\n",
              " 21,\n",
              " 10,\n",
              " 59,\n",
              " 0,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 0,\n",
              " 25,\n",
              " 22,\n",
              " 1,\n",
              " 34,\n",
              " 29,\n",
              " 22,\n",
              " 0,\n",
              " 66,\n",
              " 10,\n",
              " 62,\n",
              " 23,\n",
              " 21,\n",
              " 45,\n",
              " 28,\n",
              " 28,\n",
              " 66,\n",
              " 21,\n",
              " 26,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 52,\n",
              " 54,\n",
              " 58,\n",
              " 1,\n",
              " 28,\n",
              " 26,\n",
              " 56,\n",
              " 21,\n",
              " 59,\n",
              " 10,\n",
              " 1,\n",
              " 28,\n",
              " 22,\n",
              " 1,\n",
              " 17,\n",
              " 17,\n",
              " 10,\n",
              " 56,\n",
              " 21,\n",
              " 1,\n",
              " 26,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 0,\n",
              " 22,\n",
              " 45,\n",
              " 62,\n",
              " 22,\n",
              " 20,\n",
              " 4,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 21,\n",
              " 10,\n",
              " 59,\n",
              " 29,\n",
              " 66,\n",
              " 22,\n",
              " 66,\n",
              " 19,\n",
              " 17,\n",
              " 66,\n",
              " 28,\n",
              " 26,\n",
              " 22,\n",
              " 0,\n",
              " 65,\n",
              " 0,\n",
              " 26,\n",
              " 66,\n",
              " 2,\n",
              " 0,\n",
              " 51,\n",
              " 22,\n",
              " 0,\n",
              " 17,\n",
              " 66,\n",
              " 66,\n",
              " 21,\n",
              " 39,\n",
              " 22,\n",
              " 28,\n",
              " 66,\n",
              " 21,\n",
              " 45,\n",
              " 13,\n",
              " 34,\n",
              " 56,\n",
              " 26,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 22,\n",
              " 1,\n",
              " 34,\n",
              " 29,\n",
              " 22,\n",
              " 2,\n",
              " 1,\n",
              " 21,\n",
              " 39,\n",
              " 56,\n",
              " 34,\n",
              " 66,\n",
              " 22,\n",
              " 38,\n",
              " 56,\n",
              " 0,\n",
              " 56,\n",
              " 45,\n",
              " 34,\n",
              " 52,\n",
              " 22,\n",
              " 20,\n",
              " 28,\n",
              " 26,\n",
              " 56,\n",
              " 62,\n",
              " 56,\n",
              " 21,\n",
              " 56,\n",
              " 1,\n",
              " 10,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 26,\n",
              " 66,\n",
              " 10,\n",
              " 10,\n",
              " 56,\n",
              " 13,\n",
              " 66,\n",
              " 34,\n",
              " 21,\n",
              " 66,\n",
              " 22,\n",
              " 53,\n",
              " 20,\n",
              " 4,\n",
              " 25,\n",
              " 51,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 1,\n",
              " 27,\n",
              " 56,\n",
              " 10,\n",
              " 56,\n",
              " 26,\n",
              " 65,\n",
              " 22,\n",
              " 45,\n",
              " 62,\n",
              " 22,\n",
              " 1,\n",
              " 22,\n",
              " 29,\n",
              " 56,\n",
              " 13,\n",
              " 56,\n",
              " 26,\n",
              " 1,\n",
              " 10,\n",
              " 22,\n",
              " 21,\n",
              " 45,\n",
              " 2,\n",
              " 17,\n",
              " 59,\n",
              " 26,\n",
              " 66,\n",
              " 28,\n",
              " 22,\n",
              " 45,\n",
              " 28,\n",
              " 22,\n",
              " 21,\n",
              " 45,\n",
              " 2,\n",
              " 17,\n",
              " 59,\n",
              " 26,\n",
              " 66,\n",
              " 28,\n",
              " 23,\n",
              " 21,\n",
              " 45,\n",
              " 34,\n",
              " 26,\n",
              " 28,\n",
              " 45,\n",
              " 10,\n",
              " 10,\n",
              " 66,\n",
              " 29,\n",
              " 22,\n",
              " 28,\n",
              " 45,\n",
              " 27,\n",
              " 45,\n",
              " 26,\n",
              " 22,\n",
              " 26,\n",
              " 45,\n",
              " 22,\n",
              " 17,\n",
              " 66,\n",
              " 28,\n",
              " 62,\n",
              " 45,\n",
              " 28,\n",
              " 2,\n",
              " 22,\n",
              " 26,\n",
              " 1,\n",
              " 0,\n",
              " 46,\n",
              " 0,\n",
              " 22,\n",
              " 21,\n",
              " 45,\n",
              " 2,\n",
              " 2,\n",
              " 45,\n",
              " 34,\n",
              " 10,\n",
              " 65,\n",
              " 22,\n",
              " 1,\n",
              " 0,\n",
              " 0,\n",
              " 45,\n",
              " 21,\n",
              " 56,\n",
              " 1,\n",
              " 26,\n",
              " 66,\n",
              " 29,\n",
              " 22,\n",
              " 61,\n",
              " 56,\n",
              " 26,\n",
              " 39,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 26,\n",
              " 66,\n",
              " 10,\n",
              " 10,\n",
              " 56,\n",
              " 13,\n",
              " 66,\n",
              " 34,\n",
              " 26,\n",
              " 22,\n",
              " 27,\n",
              " 66,\n",
              " 56,\n",
              " 34,\n",
              " 13,\n",
              " 0,\n",
              " 52,\n",
              " 22,\n",
              " 11,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 26,\n",
              " 66,\n",
              " 28,\n",
              " 2,\n",
              " 22,\n",
              " 56,\n",
              " 0,\n",
              " 22,\n",
              " 62,\n",
              " 28,\n",
              " 66,\n",
              " 60,\n",
              " 59,\n",
              " 66,\n",
              " 34,\n",
              " 26,\n",
              " 10,\n",
              " 65,\n",
              " 22,\n",
              " 1,\n",
              " 17,\n",
              " 17,\n",
              " 10,\n",
              " 56,\n",
              " 66,\n",
              " 29,\n",
              " 22,\n",
              " 26,\n",
              " 45,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 17,\n",
              " 28,\n",
              " 45,\n",
              " 49,\n",
              " 66,\n",
              " 21,\n",
              " 26,\n",
              " 22,\n",
              " 45,\n",
              " 62,\n",
              " 22,\n",
              " 29,\n",
              " 66,\n",
              " 38,\n",
              " 66,\n",
              " 10,\n",
              " 45,\n",
              " 17,\n",
              " 56,\n",
              " 34,\n",
              " 13,\n",
              " 22,\n",
              " 0,\n",
              " 65,\n",
              " 0,\n",
              " 26,\n",
              " 66,\n",
              " 2,\n",
              " 0,\n",
              " 22,\n",
              " 66,\n",
              " 34,\n",
              " 29,\n",
              " 45,\n",
              " 61,\n",
              " 66,\n",
              " 29,\n",
              " 22,\n",
              " 61,\n",
              " 56,\n",
              " 26,\n",
              " 39,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 26,\n",
              " 66,\n",
              " 10,\n",
              " 10,\n",
              " 66,\n",
              " 21,\n",
              " 26,\n",
              " 59,\n",
              " 1,\n",
              " 10,\n",
              " 22,\n",
              " 17,\n",
              " 28,\n",
              " 45,\n",
              " 21,\n",
              " 66,\n",
              " 0,\n",
              " 0,\n",
              " 66,\n",
              " 0,\n",
              " 22,\n",
              " 21,\n",
              " 39,\n",
              " 1,\n",
              " 28,\n",
              " 1,\n",
              " 21,\n",
              " 26,\n",
              " 66,\n",
              " 28,\n",
              " 56,\n",
              " 0,\n",
              " 26,\n",
              " 56,\n",
              " 21,\n",
              " 22,\n",
              " 45,\n",
              " 62,\n",
              " 22,\n",
              " 39,\n",
              " 59,\n",
              " 2,\n",
              " 1,\n",
              " 34,\n",
              " 0,\n",
              " 51,\n",
              " 22,\n",
              " 0,\n",
              " 59,\n",
              " 21,\n",
              " 39,\n",
              " 22,\n",
              " 1,\n",
              " 0,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 1,\n",
              " 27,\n",
              " 56,\n",
              " 10,\n",
              " 56,\n",
              " 26,\n",
              " 65,\n",
              " 22,\n",
              " 26,\n",
              " 45,\n",
              " 22,\n",
              " 28,\n",
              " 66,\n",
              " 1,\n",
              " 0,\n",
              " 45,\n",
              " 34,\n",
              " 51,\n",
              " 22,\n",
              " 29,\n",
              " 56,\n",
              " 0,\n",
              " 21,\n",
              " 45,\n",
              " 38,\n",
              " 66,\n",
              " 28,\n",
              " 22,\n",
              " 2,\n",
              " 66,\n",
              " 1,\n",
              " 34,\n",
              " 56,\n",
              " 34,\n",
              " 13,\n",
              " 51,\n",
              " 22,\n",
              " 13,\n",
              " 66,\n",
              " 34,\n",
              " 66,\n",
              " 28,\n",
              " 1,\n",
              " 10,\n",
              " 56,\n",
              " 14,\n",
              " 66,\n",
              " 51,\n",
              " 22,\n",
              " 45,\n",
              " 28,\n",
              " 22,\n",
              " 10,\n",
              " 66,\n",
              " 1,\n",
              " 28,\n",
              " 34,\n",
              " 22,\n",
              " 62,\n",
              " 28,\n",
              " 45,\n",
              " 2,\n",
              " 22,\n",
              " 17,\n",
              " 1,\n",
              " 0,\n",
              " 26,\n",
              " 22,\n",
              " 66,\n",
              " 19,\n",
              " 17,\n",
              " 66,\n",
              " 28,\n",
              " 56,\n",
              " 66,\n",
              " 34,\n",
              " 21,\n",
              " 66,\n",
              " 52,\n",
              " 22,\n",
              " 24,\n",
              " 56,\n",
              " 34,\n",
              " 21,\n",
              " 66,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 29,\n",
              " 66,\n",
              " 38,\n",
              " 66,\n",
              " 10,\n",
              " 45,\n",
              " 17,\n",
              " 2,\n",
              " 66,\n",
              " 34,\n",
              " 26,\n",
              " 22,\n",
              " 45,\n",
              " 62,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 29,\n",
              " 56,\n",
              " 13,\n",
              " 56,\n",
              " 26,\n",
              " 1,\n",
              " 10,\n",
              " 22,\n",
              " 21,\n",
              " 45,\n",
              " 2,\n",
              " 17,\n",
              " 59,\n",
              " 26,\n",
              " 66,\n",
              " 28,\n",
              " 22,\n",
              " 56,\n",
              " 34,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 22,\n",
              " 41,\n",
              " 30,\n",
              " 9,\n",
              " 6,\n",
              " 0,\n",
              " 51,\n",
              " 22,\n",
              " 56,\n",
              " 26,\n",
              " 22,\n",
              " 39,\n",
              " 1,\n",
              " 0,\n",
              " 22,\n",
              " 27,\n",
              " 66,\n",
              " 66,\n",
              " 34,\n",
              " 22,\n",
              " 29,\n",
              " 66,\n",
              " 2,\n",
              " 45,\n",
              " 34,\n",
              " 0,\n",
              " 26,\n",
              " 28,\n",
              " 1,\n",
              " 26,\n",
              " 66,\n",
              " 29,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 1,\n",
              " 26,\n",
              " 22,\n",
              " 21,\n",
              " 45,\n",
              " 2,\n",
              " 17,\n",
              " 59,\n",
              " 26,\n",
              " 66,\n",
              " 28,\n",
              " 0,\n",
              " 22,\n",
              " 21,\n",
              " 1,\n",
              " 34,\n",
              " 22,\n",
              " 27,\n",
              " 66,\n",
              " 22,\n",
              " 17,\n",
              " 28,\n",
              " 45,\n",
              " 13,\n",
              " 28,\n",
              " 1,\n",
              " 2,\n",
              " 2,\n",
              " 66,\n",
              " 29,\n",
              " 22,\n",
              " 26,\n",
              " 45,\n",
              " 22,\n",
              " 21,\n",
              " 1,\n",
              " 28,\n",
              " 28,\n",
              " 65,\n",
              " 22,\n",
              " 45,\n",
              " 59,\n",
              " 26,\n",
              " 22,\n",
              " 38,\n",
              " 66,\n",
              " 28,\n",
              " 65,\n",
              " 22,\n",
              " 21,\n",
              " 45,\n",
              " 2,\n",
              " 17,\n",
              " 10,\n",
              " 66,\n",
              " 19,\n",
              " 22,\n",
              " 26,\n",
              " 1,\n",
              " 0,\n",
              " 46,\n",
              " 0,\n",
              " 8,\n",
              " 1,\n",
              " 0,\n",
              " 51,\n",
              " 22,\n",
              " 62,\n",
              " 45,\n",
              " 28,\n",
              " 22,\n",
              " 66,\n",
              " 19,\n",
              " 1,\n",
              " 2,\n",
              " 17,\n",
              " 10,\n",
              " 66,\n",
              " 51,\n",
              " 22,\n",
              " 29,\n",
              " 56,\n",
              " 0,\n",
              " 21,\n",
              " 45,\n",
              " 38,\n",
              " 66,\n",
              " 28,\n",
              " 56,\n",
              " 34,\n",
              " 13,\n",
              " 22,\n",
              " 17,\n",
              " 28,\n",
              " 45,\n",
              " 45,\n",
              " 62,\n",
              " 0,\n",
              " 22,\n",
              " 62,\n",
              " 45,\n",
              " 28,\n",
              " 22,\n",
              " 2,\n",
              " 1,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 2,\n",
              " 1,\n",
              " 26,\n",
              " 56,\n",
              " 21,\n",
              " 1,\n",
              " 10,\n",
              " 22,\n",
              " 26,\n",
              " 39,\n",
              " 66,\n",
              " 45,\n",
              " 28,\n",
              " 66,\n",
              " 2,\n",
              " 0,\n",
              " 22,\n",
              " ...]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F2UFBUgfn-wJ",
        "colab_type": "code",
        "outputId": "7e700611-f8dd-438e-c5fd-38c72bab7ed2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 67
        }
      },
      "source": [
        "print(f'Total sequence length of the text: {len(int_text)}')\n",
        "print(f'First 20 integers: {int_text[:20]}')\n",
        "print(f'First 20 characters: {\"\".join([int_to_char[i] for i in int_text[:20]])}')"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Total sequence length of the text: 9792\n",
            "First 20 integers: [20, 28, 26, 56, 62, 56, 21, 56, 1, 10, 22, 56, 34, 26, 66, 10, 10, 56, 13, 66]\n",
            "First 20 characters: Artificial intellige\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3PF8ysC8n-so",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def one_hot(ints, vocab_length = vocab_length):\n",
        "    outputs = nd.zeros((len(ints), vocab_length))\n",
        "    for i, idx in enumerate(ints):\n",
        "        outputs[i, idx] = 1.0\n",
        "    return outputs"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jSNFuFern-rE",
        "colab_type": "code",
        "outputId": "a1b4af6e-8686-4689-d2a8-03aa7efff30d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        }
      },
      "source": [
        "print('Before one hot:')\n",
        "print(int_text[:20])\n",
        "print('After one hot:')\n",
        "print(one_hot(int_text[:20]))"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Before one hot:\n",
            "[20, 28, 26, 56, 62, 56, 21, 56, 1, 10, 22, 56, 34, 26, 66, 10, 10, 56, 13, 66]\n",
            "After one hot:\n",
            "\n",
            "[[0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " ...\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 1.]]\n",
            "<NDArray 20x67 @cpu(0)>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VeOFs9OdqDAR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def onehot_to_text(embeddings):\n",
        "    outputs = \"\"\n",
        "    ints = nd.argmax(embeddings, axis = 1).asnumpy()\n",
        "    for i in ints:\n",
        "        outputs += int_to_char[i]\n",
        "    return outputs"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uX0Ut6_dqaRD",
        "colab_type": "code",
        "outputId": "5a3897bb-1421-404d-fee7-009898fba969",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        }
      },
      "source": [
        "onehot = one_hot(int_text[:20])\n",
        "print('First 20 characters in Onehot:')\n",
        "print(onehot)\n",
        "print('\\nAfter converting First 20 one hot embeddings to text:')\n",
        "print(f'\"{onehot_to_text(onehot)}\"')"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "First 20 characters in Onehot:\n",
            "\n",
            "[[0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " ...\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 0.]\n",
            " [0. 0. 0. ... 0. 0. 1.]]\n",
            "<NDArray 20x67 @cpu(0)>\n",
            "\n",
            "After converting First 20 one hot embeddings to text:\n",
            "\"Artificial intellige\"\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-1cIWC8aq0s5",
        "colab_type": "text"
      },
      "source": [
        "##Building the dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ihhJIxcsq4f1",
        "colab_type": "code",
        "outputId": "130c9817-fc21-44d9-c140-a8c03a2b87fa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "seq_length = 32\n",
        "num_samples = (len(int_text) - 1)//seq_length\n",
        "dataset = one_hot(int_text[: seq_length * num_samples]).reshape((num_samples, seq_length, vocab_length))\n",
        "dataset.shape"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(305, 32, 67)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z087k0tkq4bU",
        "colab_type": "code",
        "outputId": "c81f4d52-00ca-4862-e407-ae3292a201ab",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        }
      },
      "source": [
        "output = onehot_to_text(dataset[0])\n",
        "print(output)\n",
        "print(f'Sequence length: {len(output)}')"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Artificial intelligence (AI) is \n",
            "Sequence length: 32\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QrWeMXJ3q4aF",
        "colab_type": "code",
        "outputId": "fa3fa922-9ca0-4a33-ae39-ea4d0dd77a59",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        }
      },
      "source": [
        "batch_size = 8\n",
        "batches = dataset.shape[0] // batch_size\n",
        "print(f'Number of samples in dataset: {dataset.shape[0]}')\n",
        "print(f'Number of batches: {batches}')"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of samples in dataset: 305\n",
            "Number of batches: 38\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GSgXjd8Gq4ZL",
        "colab_type": "code",
        "outputId": "a73fc68c-3ebb-4ff0-d363-d9fb74bfcdb8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84
        }
      },
      "source": [
        "train_features = dataset[:batches * batch_size]\n",
        "print(train_features.shape)\n",
        "train_features = train_features.reshape((batch_size, batches, seq_length, vocab_length))\n",
        "print(train_features.shape)\n",
        "train_features = train_features.swapaxes(0, 1)\n",
        "print(train_features.shape)\n",
        "train_features = train_features.swapaxes(1, 2)\n",
        "print(train_features.shape)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(304, 32, 67)\n",
            "(8, 38, 32, 67)\n",
            "(38, 8, 32, 67)\n",
            "(38, 32, 8, 67)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TxKlVMKeq4YM",
        "colab_type": "code",
        "outputId": "60ba0ca8-fa69-4f83-e4a4-89276ea8a527",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 437
        }
      },
      "source": [
        "for i in range(5):\n",
        "    print(\"Batch %s:\\n %s \\n %s \\n\\n\" % (i, onehot_to_text(train_features[i, :, 0]), onehot_to_text(train_features[i, :, 1])))"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Batch 0:\n",
            " Artificial intelligence (AI) is  \n",
            " ks requiring much everyday knowl \n",
            "\n",
            "\n",
            "Batch 1:\n",
            " the simulation of human intellig \n",
            " edge. On the other hand, some pr \n",
            "\n",
            "\n",
            "Batch 2:\n",
            " ence processes by machines, espe \n",
            " ograms have attained the perform \n",
            "\n",
            "\n",
            "Batch 3:\n",
            " cially computer systems. These p \n",
            " ance levels of human experts and \n",
            "\n",
            "\n",
            "Batch 4:\n",
            " rocesses include learning (the a \n",
            "  professionals in performing cer \n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OxmTEWfzzrx7",
        "colab_type": "code",
        "outputId": "e7a0c3bb-d148-485a-b3c4-4d9fc174cea6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 84
        }
      },
      "source": [
        "train_labels = one_hot(int_text[1:seq_length*num_samples+1])\n",
        "print(train_labels.shape)\n",
        "train_labels = train_labels.reshape((batch_size, batches, seq_length, vocab_length))\n",
        "print(train_labels.shape)\n",
        "train_labels = nd.swapaxes(train_labels, 0, 1)\n",
        "print(train_labels.shape)\n",
        "train_labels = nd.swapaxes(train_labels, 1, 2)\n",
        "print(train_labels.shape)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(9760, 67)\n",
            "(8, 38, 32, 67)\n",
            "(38, 8, 32, 67)\n",
            "(38, 32, 8, 67)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FbMeyhCFzrql",
        "colab_type": "code",
        "outputId": "903022a8-1a9b-4f07-809c-ef00ff62226d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        }
      },
      "source": [
        "print(onehot_to_text(train_features[0, :, 4]))\n",
        "print(onehot_to_text(train_labels[0, :, 4]))"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "mmon sense, reasoning and proble\n",
            "mon sense, reasoning and problem\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g8TkdY970mKg",
        "colab_type": "text"
      },
      "source": [
        "##Initializing parameters for our model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_p8RFnKw0qI4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "num_inputs = vocab_length\n",
        "num_hidden = 512\n",
        "num_outputs = vocab_length\n",
        "\n",
        "# ==================== Unrolling in Right direction =======================\n",
        "right_Wxz = nd.random.normal(shape = (num_inputs, num_hidden), ctx = ctx) * .001\n",
        "right_Wxr = nd.random.normal(shape = (num_inputs, num_hidden), ctx = ctx) * .001\n",
        "right_Wxh = nd.random.normal(shape = (num_inputs, num_hidden), ctx = ctx) * .001\n",
        "\n",
        "right_Whz = nd.random.normal(shape = (num_hidden, num_hidden), ctx = ctx) * .001\n",
        "right_Whr = nd.random.normal(shape = (num_hidden, num_hidden), ctx = ctx) * .001\n",
        "right_Whh = nd.random.normal(shape = (num_hidden, num_hidden), ctx = ctx) * .001\n",
        "\n",
        "right_bz = nd.random.normal(shape = num_hidden, ctx = ctx) * .001\n",
        "right_br = nd.random.normal(shape = num_hidden, ctx = ctx) * .001\n",
        "right_bh = nd.random.normal(shape = num_hidden, ctx = ctx) * .001\n",
        "# =========================================================================\n",
        "\n",
        "# ==================== Unrolling in Left direction =======================\n",
        "left_Wxz = nd.random.normal(shape = (num_inputs, num_hidden), ctx = ctx) * .001\n",
        "left_Wxr = nd.random.normal(shape = (num_inputs, num_hidden), ctx = ctx) * .001\n",
        "left_Wxh = nd.random.normal(shape = (num_inputs, num_hidden), ctx = ctx) * .001\n",
        "\n",
        "left_Whz = nd.random.normal(shape = (num_hidden, num_hidden), ctx = ctx) * .001\n",
        "left_Whr = nd.random.normal(shape = (num_hidden, num_hidden), ctx = ctx) * .001\n",
        "left_Whh = nd.random.normal(shape = (num_hidden, num_hidden), ctx = ctx) * .001\n",
        "\n",
        "left_bz = nd.random.normal(shape = num_hidden, ctx = ctx) * .001\n",
        "left_br = nd.random.normal(shape = num_hidden, ctx = ctx) * .001\n",
        "left_bh = nd.random.normal(shape = num_hidden, ctx = ctx) * .001\n",
        "# =========================================================================\n",
        "\n",
        "outputs_Why = nd.random.normal(shape = (num_hidden + num_hidden, num_outputs), ctx = ctx) * .001\n",
        "outputs_by = nd.random.normal(shape = num_outputs, ctx = ctx) * .001\n",
        "\n",
        "params = [right_Wxz, right_Wxr, right_Wxh, right_Whz, right_Whr, right_Whh, right_bz, right_br, right_bh,\n",
        "          left_Wxz, left_Wxr, left_Wxh, left_Whz, left_Whr, left_Whh, left_bz, left_br, left_bh,\n",
        "          outputs_Why, outputs_by]\n",
        "\n",
        "for param in params:\n",
        "    param.attach_grad()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-Xgf_bGU1OJg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def bidirectional_gru(inputs, right_h, left_h):\n",
        "    outputs = []\n",
        "    right_hs = []\n",
        "    left_hs = []\n",
        "    \n",
        "    # unrolling in right direction =======================================================\n",
        "    for X in inputs:\n",
        "        right_z = nd.sigmoid(nd.dot(X, right_Wxz) + nd.dot(right_h, right_Whz) + right_bz)\n",
        "        right_r = nd.sigmoid(nd.dot(X, right_Wxr) + nd.dot(right_h, right_Whr) + right_br)\n",
        "        \n",
        "        right_g = nd.tanh(nd.dot(X, right_Wxh) + nd.dot(right_r * right_h, right_Whh) + right_bh)\n",
        "        right_h = right_z * right_h + (1 - right_z) * right_g\n",
        "        \n",
        "        right_hs.append(right_h)\n",
        "    \n",
        "    # unrolling in left direction =======================================================\n",
        "    if isinstance(inputs, list):\n",
        "        inputs.reverse()\n",
        "    else:\n",
        "        inputs = inputs.flip(0)\n",
        "        \n",
        "    for X in inputs:\n",
        "        left_z = nd.sigmoid(nd.dot(X, left_Wxz) + nd.dot(left_h, left_Whz) + left_bz)\n",
        "        left_r = nd.sigmoid(nd.dot(X, left_Wxr) + nd.dot(left_h, left_Whr) + left_br)\n",
        "        \n",
        "        left_g = nd.tanh(nd.dot(X, left_Wxh) + nd.dot(left_r * left_h, left_Whh) + left_bh)\n",
        "        left_h = left_z * left_h + (1 - left_z) * left_g\n",
        "        \n",
        "        left_hs.append(left_h)\n",
        "    \n",
        "    # making prdictions\n",
        "    for right_h, left_h in zip(right_hs, left_hs):\n",
        "        linear = nd.dot(nd.concat(right_h, left_h, dim = 1), outputs_Why) + outputs_by\n",
        "        output = nd.softmax(linear)\n",
        "        outputs.append(output)\n",
        "        \n",
        "    return outputs, right_hs[-1]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_ojb_FXh152M",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def generate(prefix, num_chars):\n",
        "    string = prefix\n",
        "\n",
        "    int_prefix = [char_to_int[char] for char in prefix]\n",
        "    input_sequence = one_hot(int_prefix).as_in_context(ctx)\n",
        "\n",
        "    sample_state_right = nd.zeros(shape = (1, num_hidden), ctx = ctx)\n",
        "    sample_state_left = nd.zeros(shape = (1, num_hidden), ctx = ctx)\n",
        "\n",
        "    for i in range(num_chars):\n",
        "        outputs, sample_state_right = bidirectional_gru(input_sequence, sample_state_right, sample_state_left)\n",
        "        choice = int(outputs[-1][0].argmax(0).asscalar())\n",
        "        string += int_to_char[choice]\n",
        "        input_sequence = one_hot([choice]).as_in_context(ctx)\n",
        "    return string"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L9bctEbC6sY6",
        "colab_type": "code",
        "outputId": "607d55a4-bf83-46bd-e6fa-99010ce091cb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "print(generate(\"Artificial intelligence\", 100))"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Artificial intelligencehhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UJbClx0_1OGd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def cross_entropy(pred, label):\n",
        "    return -nd.sum(label * nd.log(pred))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WHjdxaUO_2Vj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def objective(outputs, labels):\n",
        "    total_loss = 0.\n",
        "    for (output, label) in zip(outputs,labels):\n",
        "        total_loss = total_loss + cross_entropy(output, label)\n",
        "    return total_loss / len(outputs)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YRJTLBco1OEg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def sgd(params, lr):\n",
        "    for param in params:\n",
        "        param[:] = param - lr * param.grad"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WlmjfU8B1N_x",
        "colab_type": "code",
        "outputId": "1415492f-ac70-47bf-a746-f689bd1ceab5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "epochs = 100\n",
        "learning_rate = .5\n",
        "losses = []\n",
        "\n",
        "for epoch in range(epochs):\n",
        "    if (epoch + 1) % 40 == 0:\n",
        "        learning_rate = learning_rate / 2.0\n",
        "    \n",
        "    state_right = nd.zeros(shape = (batch_size, num_hidden), ctx = ctx)\n",
        "    state_left = nd.zeros(shape = (batch_size, num_hidden), ctx = ctx)\n",
        "    \n",
        "    for i in range(batches):\n",
        "        features = train_features[i].as_in_context(ctx)\n",
        "        labels = train_labels[i].as_in_context(ctx)\n",
        "        with autograd.record():\n",
        "            outputs, _ = bidirectional_gru(features, state_right, state_left)\n",
        "            loss = objective(outputs, labels)\n",
        "            loss.backward()\n",
        "        sgd(params, learning_rate)\n",
        "    \n",
        "    losses.append(loss.asscalar())\n",
        "    print(f'\\nEpoch: {epoch} | Loss: {loss.asscalar():.5f}')\n",
        "    print(generate(\"Artificial intelligence\", 100))\n",
        "    print('-'*50)\n",
        "    print(generate(\"Research in AI has focused chiefly on\", 100))"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Epoch: 0 | Loss: 24.50465\n",
            "Artificial intelligence                                                                                                    \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on                                                                                                    \n",
            "\n",
            "Epoch: 1 | Loss: 23.58868\n",
            "Artificial intelligence t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in t in\n",
            "\n",
            "Epoch: 2 | Loss: 21.56823\n",
            "Artificial intelligence te te te te te te te te te te te te te te te te te te te te te te te te te te te te te te te te te \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on te te te te te te te te te te te te te te te te te te te te te te te te te te te te te te te te te \n",
            "\n",
            "Epoch: 3 | Loss: 21.19427\n",
            "Artificial intelligence the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "\n",
            "Epoch: 4 | Loss: 19.96942\n",
            "Artificial intelligence the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "\n",
            "Epoch: 5 | Loss: 19.44921\n",
            "Artificial intelligence the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "\n",
            "Epoch: 6 | Loss: 18.98103\n",
            "Artificial intelligence the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "\n",
            "Epoch: 7 | Loss: 18.61847\n",
            "Artificial intelligencengengengengengengengengengengengengengengengengengengengengengengengengengengengengengengengengengen\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "\n",
            "Epoch: 8 | Loss: 18.26133\n",
            "Artificial intelligencenengerengerengerengerengerengerengerengerengerengerengerengerengerengerengerengerengerengerengerenge\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on therengengengerengerengerengerengerengerengerengerengerengerengerengerengerengerengerengerengerenge\n",
            "\n",
            "Epoch: 9 | Loss: 18.01397\n",
            "Artificial intelligencenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenene\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the the the the the the the the the the the the the the the the the the the the the the the the the\n",
            "\n",
            "Epoch: 10 | Loss: 17.75245\n",
            "Artificial intelligencenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenene\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on interenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenen\n",
            "\n",
            "Epoch: 11 | Loss: 17.26631\n",
            "Artificial intelligence the interenenerenerenerenerenerenerenerenerenerenerenerenerenerenerenerenerenerenerenerenerenerener\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on interenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenenen\n",
            "\n",
            "Epoch: 12 | Loss: 16.83113\n",
            "Artificial intelligence the of interention in the interention in the interention in the interention in the interention in t\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the interention in the interention in the interention in the interention in the interention in the \n",
            "\n",
            "Epoch: 13 | Loss: 16.49587\n",
            "Artificial intelligence and in the in the in the in the in the in the in the in the in the in the in the in the in the in t\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the interenting in the in the in the in the in the in the in the in the in the in the in the in the\n",
            "\n",
            "Epoch: 14 | Loss: 16.24944\n",
            "Artificial intelligence and in the interenting in the interenting in the interenting in the interenting in the interenting \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the sing in the sing in the sing in the sing in the sing in the sing in the sing in the sing in the\n",
            "\n",
            "Epoch: 15 | Loss: 16.01820\n",
            "Artificial intelligence and intelligent of the sint in the sint in the sint in the sint in the sint in the sint in the sint\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the sint in the sint in the sint in the sint in the sint in the sint in the sint in the sint in the\n",
            "\n",
            "Epoch: 16 | Loss: 15.76396\n",
            "Artificial intelligence and intelligence and intelligence and intelligence and intelligence and intelligence and intelligen\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the sine in the intelligence and intelligence and intelligence and intelligence and intelligence an\n",
            "\n",
            "Epoch: 17 | Loss: 15.50363\n",
            "Artificial intelligence and intelligence and intelligence and intelligence and intelligence and intelligence and intelligen\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the intelligence and intelligence and intelligence and intelligence and intelligence and intelligen\n",
            "\n",
            "Epoch: 18 | Loss: 15.24383\n",
            "Artificial intelligence and intelligence and intelligence and intelligence and intelligence and intelligence and intelligen\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the and intelligence and intelligence and intelligence and intelligence and intelligence and intell\n",
            "\n",
            "Epoch: 19 | Loss: 15.05387\n",
            "Artificial intelligence and intelligence and intelligence and intelligence and intelligence and intelligence and intelligen\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the and intelligence and intelligence and intelligence and intelligence and intelligence and intell\n",
            "\n",
            "Epoch: 20 | Loss: 14.34721\n",
            "Artificial intelligence and intelligence and intelligence and intelligence and intelligence and intelligence and intelligen\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the problemm the problemm the problemm the problemm the problemm the problemm the problemm the prob\n",
            "\n",
            "Epoch: 21 | Loss: 13.90040\n",
            "Artificial intelligence and intelligence of in the problems in the and intelligence of in the problems in the and intellige\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the with the problems in the and intelligence of in the problems in the and intelligence of in the \n",
            "\n",
            "Epoch: 22 | Loss: 14.52659\n",
            "Artificial intelligence of intelligence of intelligence of intelligence of intelligence of intelligence of intelligence of \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly ontenting intelligence of intelligence of intelligence of intelligence of intelligence of intelligence\n",
            "\n",
            "Epoch: 23 | Loss: 13.25203\n",
            "Artificial intelligence and intelligence and intelligence and intelligence and intelligence and intelligence and intelligen\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the problems in the problems of in the problems in the problems of in the problems in the problems \n",
            "\n",
            "Epoch: 24 | Loss: 13.24188\n",
            "Artificial intelligence and intelligence of intelligence of intelligence of intelligence of intelligence of intelligence of\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the problems intelligence and intelligence of intelligence of intelligence of intelligence of intel\n",
            "\n",
            "Epoch: 25 | Loss: 12.63766\n",
            "Artificial intelligence in the problems in tre are or and on in to diveroning in tre are or and on in to diveroning in tre \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on tre are or and on in to diveroning in tre are or and on in to diveroning in tre are or and on in to\n",
            "\n",
            "Epoch: 26 | Loss: 12.18911\n",
            "Artificial intelligence is comation and intelligence in the problem in the problem in the problem in the problem in the pro\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on tre sere can the case of in the case of in the case of in the case of in the case of in the case of\n",
            "\n",
            "Epoch: 27 | Loss: 11.93041\n",
            "Artificial intelligence problems inteding can beere program the pastite inceuse AI seneering inted inteding can beere and m\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on infice and sinee on infice and sineer past presing inceure AI sere can beeneralize computer sting i\n",
            "\n",
            "Epoch: 28 | Loss: 11.61423\n",
            "Artificial intelligence present on in the simple mast be in example rogoine AI in the siences in the sole conning AI with d\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on in of in mation. This is an on in to diver in the solection in in of in mede and in the sience wisc\n",
            "\n",
            "Epoch: 29 | Loss: 11.29634\n",
            "Artificial intelligence as deenoral fo many of in the haratere the unally th the mast of in the situal in inductive in the \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on slingliticlligence as ditforemmming th the situal intelligence as ditforemmming th the situal intel\n",
            "\n",
            "Epoch: 30 | Loss: 10.46624\n",
            "Artificial intelligence as dimilation invile ondution in to norvelation. This is one and of is as nompreentell goneraine wi\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on troulation. This is an of arility ofteneralized to inallogs sorvine in carable of intile intelligen\n",
            "\n",
            "Epoch: 31 | Loss: 9.60780\n",
            "Artificial intelligence is in in in in tasks sulled. Thileracts.\n",
            "Thin to caractelite currn with grewience. AI trained to AI\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on trainele or in tasks—sumaning withous in tre care as an mate on ir is is mance the andenition in tr\n",
            "\n",
            "Epoch: 32 | Loss: 9.02775\n",
            "Artificial intelligence withous in cilllicidg AI in taskncateditian onten action. Initialscient in cillloging AI in the hum\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the wirron on the andering wiscosing computer many ne cleduct the inall on inllicied into dew reasi\n",
            "\n",
            "Epoch: 33 | Loss: 8.91101\n",
            "Artificial intelligence must beiononsiduren.\n",
            "Inserensing inferences are inieity of indumanies on inllems confronning An tor\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on indumaticat ininilions in imanns of inalligence of situations in implemen to inrommont invill in im\n",
            "\n",
            "Epoch: 34 | Loss: 9.04574\n",
            "Artificial intelligence mathinition in to in a lare of theit an experint callegnies appropricater the endicial intelligence\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the ano nith of the wircts. There are task kere or in tisk a or in to a carrention on the are are o\n",
            "\n",
            "Epoch: 35 | Loss: 9.05919\n",
            "Artificial intelligence. AI can learnien.\n",
            "In to AIIPPonolles of AI indumanient will deduction and in to inference withoud i\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the larelation AI intelligence. AI croblem-on andmming is condusting inverving involves compleing A\n",
            "\n",
            "Epoch: 36 | Loss: 7.73232\n",
            "Artificial intelligence must hared. To releare of experies ena no verr wier a bender in the have objects, Maching. That is \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the ability tooblem of experience. Sinces.\n",
            "There is iner to nouction and mapried to inverenting par\n",
            "\n",
            "Epoch: 37 | Loss: 7.92923\n",
            "Artificial intelligence must have objen as access in clare of just ble aboly drawinin of irrificially on illlomming conside\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the induction and mighing of in im also k of nuch is commming of such as for a sity on s in trainin\n",
            "\n",
            "Epoch: 38 | Loss: 7.21515\n",
            "Artificial intelligence must havious is inalloud on the ability object may appear diminition of in to kenion. Thie to the d\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the ability objects in the scene is an of the liean oblet on the ability object may appear diminiti\n",
            "\n",
            "Epoch: 39 | Loss: 3.74392\n",
            "Artificial intelligence must have and mapping.\n",
            "To reason is to drawing AI cloud offerings include the ability to as applica\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the ability to applace the endict to haw intelligence. What is the experience. Since the develope t\n",
            "\n",
            "Epoch: 40 | Loss: 2.84220\n",
            "Artificial intelligence must have abuple experience. Since the deveron diefore to the case object matering in vill is the d\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the ability to as apted to generally do not ringred to arainied. In procled to propused intelligenc\n",
            "\n",
            "Epoch: 41 | Loss: 2.26223\n",
            "Artificial intelligence must have abuent w program that learns—sutrod with a bumpone mealoning infommation to aninitive acc\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the ability to ass to derence the past tense of mathematical isplems and simple mapping.\n",
            "To reason \n",
            "\n",
            "Epoch: 42 | Loss: 2.00988\n",
            "Artificial intelligence must have abuple experied the problem is empecapl, a derensidion im will hand refuctive. Seneealize\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the acommation the invermanienche and machine learning as applical computer science. the past tense\n",
            "\n",
            "Epoch: 43 | Loss: 1.69593\n",
            "Artificial intelligence must have abudint of the porlledution without human intervention and mappine. To epperience. Since \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the actelligence mathide can le currow while inteering developed to or intrudent larign. There bero\n",
            "\n",
            "Epoch: 44 | Loss: 1.60895\n",
            "Artificial intelligence must haviour is an AI system that past experimunt with an unfamil and maphied if the case of a simu\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the actell, digfers and machine learning as applical computer science often restamp, computer pract\n",
            "\n",
            "Epoch: 45 | Loss: 1.41475\n",
            "Artificial intelligence must have abudint to that difference behaviour of the digference between the current human clessers\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the actelligence (A whol hata or about on the forelt do intelligence as either weak nearnce. What i\n",
            "\n",
            "Epoch: 46 | Loss: 1.37416\n",
            "Artificial intelligence must have abuplace the hurrt categorine and precial computer science often reseation. Pnowledg gene\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the actelligent beinnss data problems such as ned situation. Ploblem solving, particular purpose me\n",
            "\n",
            "Epoch: 47 | Loss: 1.35222\n",
            "Artificial intelligence must haviour is a core thery humpudation in the scene is decomposed into seppedings. The learn the \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the late algogrimine the warepss of intelligence must haviour is as naw rea charactedi, algo the wi\n",
            "\n",
            "Epoch: 48 | Loss: 1.25545\n",
            "Artificial intelligence must have abudint to that computational learning as application involves applace and lige in genera\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the actelligence (utw, mumhied of the larter, “ore hard solving method digide generalization. Gools\n",
            "\n",
            "Epoch: 49 | Loss: 1.14403\n",
            "Artificial intelligence must haviour is as berawnn as eaniratlod AI for various sppect intelligence in this limide furing s\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the form tasks such as facialign. cherplo on AItificine and the care lied duappn.int that an object\n",
            "\n",
            "Epoch: 50 | Loss: 1.09417\n",
            "Artificial intelligence in this limige. computer seccection poofomm tasssis, ta dreg as a Service allows individuals repetc\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and reralcion hita is the conclusion without gineralize. There allo in tre\n",
            "\n",
            "Epoch: 51 | Loss: 1.07656\n",
            "Artificial intelligence must have abudint of durpose teh the simple tre huramming cuttus diccgsitian be can be problem sump\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and rrogral gence, may be charning is a core Surrow while she is inside: o\n",
            "\n",
            "Epoch: 52 | Loss: 1.09315\n",
            "Artificial intelligence. may as e term the category an object belongs the past tense of regular EI.\n",
            "The infermnces a progra\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation statis in claselysing computers to draw inferences appropriat\n",
            "\n",
            "Epoch: 53 | Loss: 1.02025\n",
            "Artificial intelligence in this limigation. This is one of the hardest to bet experience with similar verbs.\n",
            "P processen co\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the acherion dial goouspe try capectally reter profems endoreditial intellicen between the ference \n",
            "\n",
            "Epoch: 54 | Loss: 1.01343\n",
            "Artificial intelligence must haviour is as bed computer program for solving computer science often requiren as a Service al\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the larter, “orion dial geal is a well-defined branch of theo AI camare oulatien. Rhis is com basin\n",
            "\n",
            "Epoch: 55 | Loss: 0.98323\n",
            "Artificial intelligence must haviour is as bedwin processes bean nowstate and the final goal. The pirticular task or situat\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation of muth diefic teal ietuen. Thiepation in the scene is decomp\n",
            "\n",
            "Epoch: 56 | Loss: 0.96107\n",
            "Artificial intelligence must haviour is as bedw wotie,, the abiett or setron hith sube berowen and mmoted is a well-defined\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation of muth dovisien.\n",
            "In performance the scans and sulporite husb\n",
            "\n",
            "Epoch: 57 | Loss: 0.95534\n",
            "Artificial intelligence must haviour is as bedwin is inclute ehempaning of the world, whore professionals in performing cer\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation of muth ov trean serffccal geal not as applications for of th\n",
            "\n",
            "Epoch: 58 | Loss: 0.93725\n",
            "Artificial intelligence must haviour is as bedwin is include e problem the postorrection.\n",
            "Thie as applications in a to dede\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation of suthaal weas nere tral kellogences relevant to the solutio\n",
            "\n",
            "Epoch: 59 | Loss: 0.92445\n",
            "Artificial intelligence must haviour is as bedwin is include e problem the postorrection.\n",
            "Thie as applications in a to dede\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation of suthaal weas nere tral kell for uitide. The poolligencen l\n",
            "\n",
            "Epoch: 60 | Loss: 0.91269\n",
            "Artificial intelligence (AI) is to beratt th thetion. Phis is one of the hardest to bet experience with similar verbs.\n",
            "P pr\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation of muth ov tre to an object belonnsit is clas, dirploces and \n",
            "\n",
            "Epoch: 61 | Loss: 0.90089\n",
            "Artificial intelligence must haviour is as bedwin is include ehat the fore. infremented rogret the category an object belon\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation of muth ov tre to an object belonnsitua plocedures—ks ar inal\n",
            "\n",
            "Epoch: 62 | Loss: 0.88883\n",
            "Artificial intelligence (AI) is to brain staffin to behaviour is ascribed to intelligence is too closely limalsis—is comput\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relligence— AI applace abilified the partitular task or situation. Inf\n",
            "\n",
            "Epoch: 63 | Loss: 0.87730\n",
            "Artificial intelligence (AI) is to l rel to the conclution with generalization. Generalization. Generalization. Generalizat\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relligence— may hame, to dea nelvined computer encountered the same po\n",
            "\n",
            "Epoch: 64 | Loss: 0.86352\n",
            "Artificial intelligence (AI) is to brain staffin to behaviour is ascribed to intelligence is too closely limalsis—is compun\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relligence— may hame, to dea nelvined computer , corectsten and mappin\n",
            "\n",
            "Epoch: 65 | Loss: 0.85585\n",
            "Artificial intelligence (AI) is to brain staffin to behaviour is ascribed to intelligence is too closely limalsis—is compun\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relligence— may be charainitias ot intelligence. WAac is, ther deapsin\n",
            "\n",
            "Epoch: 66 | Loss: 0.84252\n",
            "Artificial intelligence (AI) is to brain staffin to behaviour is ascribed to intelligence is too closely limalsis—is compun\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relligence— may hame, to dea nelvined computer , comply to a wern nomp\n",
            "\n",
            "Epoch: 67 | Loss: 0.83388\n",
            "Artificial intelligence (AI) is to brain stfffr mash and to analyce sity of irepparistate onden as artificial goal geals pu\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relligence— may be charainitias ot intelligence. WAap ans common earac\n",
            "\n",
            "Epoch: 68 | Loss: 0.82204\n",
            "Artificial intelligence (AI) is to brain stfffr mash and to analyce sity of irepparate objects, categories, humpe the dacti\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relligence— may be charainitias ot intelligence. WAap ans common earac\n",
            "\n",
            "Epoch: 69 | Loss: 0.81302\n",
            "Artificial intelligence (AI) is to brain stfffr mash and to analyce sity of irepparistate or drogris dising sssiffenst adga\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relliced or intrlleducl sen chiefic tnat in ot a Sind al ofler with an\n",
            "\n",
            "Epoch: 70 | Loss: 0.80138\n",
            "Artificial intelligence (AI) is to brain soffin the cafact to hava or nerritide ability to rigeden on the most advanced AI \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components with aboat confiter peasonnsis a reasoning and caricie, hhal detuction and\n",
            "\n",
            "Epoch: 71 | Loss: 0.80796\n",
            "Artificial intelligence must havion to abiett of nuch a sonving generalization so fridure a ford that computers can be prob\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and regresticl widp suete object the general publi. More thal hive stbe mu\n",
            "\n",
            "Epoch: 72 | Loss: 0.77044\n",
            "Artificial intelligence (AI) is to brain to husbucatedgare an objeciss if clese using renarintedity top procedure as a prod\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following components and relation of theowawin dasbo masiem in the case of a simullous ire selo\n",
            "\n",
            "Epoch: 73 | Loss: 1.03397\n",
            "Artificial intelligence must have and profess enmssing invorve abel. Rose stromm thereftrated to Arterited to are is det fo\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the late llagg thgogh artificial intelligence must have and profess enmssing invorve abelytik in th\n",
            "\n",
            "Epoch: 74 | Loss: 10.64967\n",
            "Artificial intelligence must be in eitheremmon defened while if the experimine to a curproblems of intramm the solution in \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the limad the ability the empredict to didenerito dre iner to are tre tre haddede enfenied the simp\n",
            "\n",
            "Epoch: 75 | Loss: 4.72886\n",
            "Artificial intelligence as approbable to generalization. This is one offerings, as applications inderent defene numprititua\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the formerion her foouthout hand with simullem leagning chtss fictiffichat is also a mojet toiks an\n",
            "\n",
            "Epoch: 76 | Loss: 3.39012\n",
            "Artificial intelligence and infersn ass to developing so e asonications is a more that an object manipulation and naving on\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the scene processes include the acility to AI tools present a range of possiclongly asseen enporonc\n",
            "\n",
            "Epoch: 77 | Loss: 2.26594\n",
            "Artificial intelligence must have abundant to the difference between the current of the digital computerns on objectsmen co\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the ability to applaratt ot is in trencaused by instruned II computers, ande enalynoss to AI wirvor\n",
            "\n",
            "Epoch: 78 | Loss: 1.54298\n",
            "Artificial intelligence must havil to applarant the nawlled to objects, categories, hor example, discover many the scene is\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the angll must have abundant inferences appropriate to the care; the café. Ho here are a Service an\n",
            "\n",
            "Epoch: 79 | Loss: 0.94816\n",
            "Artificial intelligence must have abundant informational relet to dewered and theian past tensy of rugeearing of problems. \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componed with the posed.\n",
            "Because hardware, software and the final goal. The program t\n",
            "\n",
            "Epoch: 80 | Loss: 0.83541\n",
            "Artificial intelligence must haviour is reveal, wasp net rnomplicat diave the digetral an object manipulation and navions o\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen to part that al object manipulation and navions of a more neutral computers \n",
            "\n",
            "Epoch: 81 | Loss: 0.81103\n",
            "Artificial intelligence must haviour is revealy past experience. Since the deverone human intervention and the fore a latio\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen to part that al object aning is common in mathematics and mholly doss to AI \n",
            "\n",
            "Epoch: 82 | Loss: 0.79364\n",
            "Artificial intelligence must haviour is revealy past experience. Since the deverone human ffiefine and mpecedvel product to\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of regular EI system that uner\n",
            "\n",
            "Epoch: 83 | Loss: 0.78003\n",
            "Artificial intelligence must haviour is revealy wasp returns on the angle from which it is the coaction and intensing (ust \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of regular EI system that uner\n",
            "\n",
            "Epoch: 84 | Loss: 0.76637\n",
            "Artificial intelligence must haviour is revealy wasp ane the digetral an object manied. Inferences are class to the posto r\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of regular EI system that uner\n",
            "\n",
            "Epoch: 85 | Loss: 0.75474\n",
            "Artificial intelligence must haviour is revealy wasp ane the digetral an object manied. Inferences are class to the postort\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of regular EI system that uner\n",
            "\n",
            "Epoch: 86 | Loss: 0.74449\n",
            "Artificial intelligence must haviour is reveal, what AI system t appacide information and rule and so forms and procedures—\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of regular EI system that uner\n",
            "\n",
            "Epoch: 87 | Loss: 0.73533\n",
            "Artificial intelligence must haviour is reveal, what AI system t appacide information and rule and so forms and procedures—\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of regular EI system that uner\n",
            "\n",
            "Epoch: 88 | Loss: 0.72710\n",
            "Artificial intelligence must haviour is reveal, what AI system t appacide information and rule and so forms and procedures—\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of regular EI system that uner\n",
            "\n",
            "Epoch: 89 | Loss: 0.71958\n",
            "Artificial intelligence must haviour is reveal, what AI system t appacide ifferent an abintes and program that learning (it\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of regular EI system that uner\n",
            "\n",
            "Epoch: 90 | Loss: 0.71265\n",
            "Artificial intelligence must haviour is reveal, what AI system t appacide ifferent an abintes and program that learning (it\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of jump based on the threshold\n",
            "\n",
            "Epoch: 91 | Loss: 0.70621\n",
            "Artificial intelligence must haviour is reveal, where has beeped an object manipulation and navion of intelligence: learnin\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of jump based on the threshold\n",
            "\n",
            "Epoch: 92 | Loss: 0.70015\n",
            "Artificial intelligence must haviour is reveal, where has beeped an object manipulation and navion of intelligence: learnin\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of jump based on the threshold\n",
            "\n",
            "Epoch: 93 | Loss: 0.69443\n",
            "Artificial intelligence must haviour is reveal, where has beepeand the problem is embedded. Intelligence—at is an AI system\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of jump based on the threshold\n",
            "\n",
            "Epoch: 94 | Loss: 0.68900\n",
            "Artificial intelligence must haviour is reveal, where has beepeand the problem is embedded. Intelligence—at is an AI system\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of jump based on the threshol,\n",
            "\n",
            "Epoch: 95 | Loss: 0.68380\n",
            "Artificial intelligence must haviour is reveal, where has beepeand the problem is embedded. Intelligence—at is an AI system\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs the past tense of jump based on the threshol,\n",
            "\n",
            "Epoch: 96 | Loss: 0.67882\n",
            "Artificial intelligence must haviour is reveal, where has beepean past experience. Since the deverve abued to fand gese of \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs th the porented with the posed to topuslapac \n",
            "\n",
            "Epoch: 97 | Loss: 0.67402\n",
            "Artificial intelligence must haviour is reveal, where has beepean past experience. Since the deverve abued to fand gese of \n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs th the porented with the posed to topuslapac \n",
            "\n",
            "Epoch: 98 | Loss: 0.66937\n",
            "Artificial intelligence must haviour is ascribed to intelligence and the ability to reason, discover mayike of ehamples, th\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs th the porented with the posed to topusla as \n",
            "\n",
            "Epoch: 99 | Loss: 0.66487\n",
            "Artificial intelligence must haviour is ascribed to intelligence and the ability to reason, discover mayike of ehamples, th\n",
            "--------------------------------------------------\n",
            "Research in AI has focused chiefly on the following componen the category an object belongs th the porented with the posed to applarate o\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e51niRQbF0AL",
        "colab_type": "code",
        "outputId": "30f66a89-a8a0-4811-f71b-fabd4adf62d7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281
        }
      },
      "source": [
        "plt.plot(losses, c = 'r')\n",
        "plt.title('Training Loss')\n",
        "plt.show()"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmUVNW5/vHv2wODzEjLKGKckDig\n9jUqXjVoFBwgGr3JVQxGDRp1OcQ4JDG5mVZijHGMl4hBJcY4x0QFRcQp+ovGxquCgrMoiNKijA3Y\n0O/vj31Kqptuurq7qs6pquezVq2qOnXKeo9Hn9q9a5+9zd0REZHCVxZ3ASIikh0KdBGRIqFAFxEp\nEgp0EZEioUAXESkSCnQRkSKhQJeCZGblZrbazIZmc1+RQmYahy75YGar055uBawHNkbPz3D32/Nf\nVceZ2a+AIe5+Sty1iFTEXYCUBnfvnnpsZu8Bp7v7Yy3tb2YV7r4hH7WJFAt1uUgimNmvzOwuM7vD\nzFYBE8xsfzN7zsyWm9kSM7vOzCqj/SvMzM1sWPT8L9HrD5vZKjP7l5lt39Z9o9fHmtkbZrbCzK43\ns2fN7JR2HNOXzeypqP65ZnZU2mtHm9n86PMXmdkF0fZtzGxG9J5Pzezp9v47ldKjQJckORb4K9AL\nuAvYAJwH9ANGAWOAM7bw/hOBnwB9gfeBX7Z1XzPbBrgbuCj63HeBfdt6IGbWCXgImA5UARcAd5nZ\njtEutwCnuXsPYA/gqWj7RcA70XsGAJe19bOldCnQJUmecfcH3b3B3de6+wvu/ry7b3D3d4ApwMFb\neP+97l7j7vXA7cDIdux7NPCSu/8jeu1q4JN2HMsooBPwO3evj7qXHga+Fb1eD4wwsx7u/qm7v5i2\nfRAw1N0/d3e10CVjCnRJkg/Sn5jZcDObbmYfmdlK4BeEVnNLPkp7XAd0b2nHLew7KL0OD6MGFmVQ\ne1ODgPe98aiDhcDg6PGxwDjgfTN70sy+Em2/PNpvtpm9bWYXteOzpUQp0CVJmg65uhGYB+zo7j2B\nnwKW4xqWAENST8zM2BTCbfEhsG30/pShwGKA6C+PccA2hK6ZO6PtK939AncfBnwduMTMtvRXicgX\nFOiSZD2AFcAaM9uVLfefZ8tDwN5mdoyZVRD68KtaeU+5mXVJu3UG/h/hN4ALzazSzEYDRxL60bua\n2Ylm1jPq1lkFNABEn7tD9EWwgjC0syE3hyrFRoEuSXYhMJEQeDcSfijNKXf/GPgmcBWwDNgB+D/C\nuPmWTADWpt1ed/f1wDHAeEIf/HXAie7+ZvSeicDCqCvptOifAbAL8DiwGngWuNbd/5m1A5SipguL\nRLbAzMoJ3SfHK1gl6dRCF2nCzMaYWe+o6+QnhJEn/465LJFWKdBFNncgYSx4LXAEcGzUhSKSaOpy\nEREpEmqhi4gUibxOztWvXz8fNmxYPj9SRKTgzZkz5xN3b234bH4DfdiwYdTU1OTzI0VECp6ZLcxk\nP3W5iIgUiVYD3cy2NbMnzOw1M3vVzM6Ltv/MzBab2UvR7cjclysiIi3JpMtlA3Chu79oZj2AOWY2\nK3rtane/MnfliYhIploNdHdfQpiwCHdfZWbzad9kRSIikkNt6kOPVnzZC3g+2nSOmb1iZjebWZ8W\n3jPJzGrMrKa2trZDxYqISMsyDnQz6w7cB5zv7iuByYSJi0YSWvC/b+597j7F3avdvbqqqtVRNyIi\n0k4ZBXq0juN9wO3u/jcIs9K5+0Z3bwBuoh3LdImISPZkMsrFgKnAfHe/Km37wLTdjiUsRJAbs2bB\n5Zfn7B8vIlIMMmmhjwJOBkY3GaJ4RbSS+SvAVwmL4ObGrFlw2WXw4Yc5+wgRkULXaqC7+zPubu6+\nh7uPjG4z3P1kd9892j4uGg2TG5MmwcaNcPPNOfsIEZFCVxhXiu64Ixx2GEyZEoJdREQ2UxiBDnDm\nmfDBB/DII3FXIiKSSIUT6OPGwYAB8Mc/xl2JiEgiFU6gV1bCaafBjBnw/vtxVyMikjiFE+gA3/0u\nuMOf/hR3JSIiiVNYgb7ddjB2bAh0LZ0nItJIYQU6hL70JUtgYUbzvYuIlIzCC/Q99wz3L78cbx0i\nIglTeIG+++5gpkAXEWmi8AK9W7dwoZECXUSkkcILdAjdLgp0EZFGCjfQ334bVq2KuxIRkcQo3EAH\nmDs33jpERBKksANd3S4iIl8ozEDfdlvo3VuBLiKSpjAD3Uw/jIqINFGYgQ4h0OfOhYaGuCsREUmE\nwg70NWvCaBcRESnwQAd1u4iIRAo30L/8ZSgrU6CLiEQKN9C7dIFddlGgi4hECjfQQSNdRETSFHag\n77NPWI7uH/+IuxIRkdgVdqBPmgT77gsnnAAPPhh3NSIisSrsQO/ZE2bODF0v3/gGPPRQ3BWJiMSm\nsAMdwhQAjz4Ke+wRQn3RorgrEhGJReEHOkCfPvCXv8Dnn6s/XURKVnEEOsDw4bDzzvDAA3FXIiIS\ni+IJdIBjjoEnnoCVK+OuREQk74ov0OvrQ5+6iEiJKa5AHzUq9KdrCKOIlKDiCvSKCjjySJg+HTZu\njLsaEZG8ajXQzWxbM3vCzF4zs1fN7Lxoe18zm2Vmb0b3fXJfbgbGjYNly+Bf/4q7EhGRvMqkhb4B\nuNDdRwD7AWeb2QjgUmC2u+8EzI6ex++II0JLXaNdRKTEtBro7r7E3V+MHq8C5gODgfHAtGi3acDX\nc1Vkm/TqBYccon50ESk5bepDN7NhwF7A80B/d18SvfQR0D+rlXXEMcfAggXw4otxVyIikjcZB7qZ\ndQfuA85390YDvd3dAW/hfZPMrMbMampraztUbMZOPBG22QYmToR16/LzmSIiMcso0M2skhDmt7v7\n36LNH5vZwOj1gcDS5t7r7lPcvdrdq6uqqrJRc+v69YNbb4V58+DSZHTti4jkWiajXAyYCsx396vS\nXnoAmBg9nggkaxKVsWPh3HPh2mvhkUfirkZEJOcs9JZsYQezA4F/AnOBhmjzjwj96HcDQ4GFwH+5\n+6db+mdVV1d7TU1NR2vO3Lp18B//AbW1YWWj/snp5hcRyZSZzXH36tb2q2htB3d/BrAWXj60rYXl\nVZcu8Ne/hkUwvvlNmDULKivjrkpEJCeK60rR5uy+O9x0Ezz1FFx8cdzViIjkTKst9KIwYQK88AJc\nc03ogjnxxLgrEhHJuuJvoadceSX853/C6aeHcBcRKTKlE+iVlXDPPeGH0cMOg+eei7siEZGsKp1A\nhxDmTz0FVVVw+OHwzDNxVyQikjWlFegAQ4eGUB84EMaMgSefjLsiEZGsKL1ABxg8OIT6dtuFC5Bm\nzoy7IhGRDivNQAcYMCC0zocPD3Ooa7pdESlwpRvoEPrSH38cRo6Eb3wDJk+GVq6cFRFJqtIOdAhr\nkM6aBaNHw1lnhR9LFy6MuyoRkTZToAP07Bkm8Jo8OQxn3H13mDpVrXURKSgK9BQzOPNMmDs3XE16\n+ulw3HHwySdxVyYikhEFelPDhoUumCuvhBkzQmv96afjrkpEpFUK9OaUlcGFF8K//x3WKB0/Ht59\nN+6qRES2SIG+JXvuGVrp7nDCCbB+fdwViYi0SIHemi99CaZNgzlz4IIL4q5GRKRFCvRMjB8PP/hB\nGAVz++1xVyMi0iwFeqZ+/etN0+8+/3zc1YiIbEaBnqnKSrjvvjCp17hx8N57cVckItKIAr0tqqpg\n+vTw4+jRR8OKFXFXJCLyBQV6W+26a2ipv/56uPBo1aq4KxIRARTo7XPooXDLLWEK3tGjYenSuCsS\nEVGgt9uECfD3v8Orr8KoUfDOO3FXJCIlToHeEUcfDbNnw7JlYf6XO+7QhF4iEhsFekftv3+YoXHH\nHeHEE8O86h99FHdVIlKCFOjZsPPO8OyzcMUVYaqAESPC9LsNDXFXJiIlRIGeLRUVcNFF8NJLsNtu\n4QKkgw+G116LuzIRKREK9GwbPjysVTp1agjzkSPht7+FjRvjrkxEipwCPRfKyuDUU2HBgnBV6aWX\nwkEHwVtvxV2ZiBQxBXouVVXBPffAX/4ShjfuuSdMmaKRMCKSEwr0XDODk06CefPggAPgjDPC7I26\nGElEskyBni9DhsDMmXD11fDoo2Fpu4ceirsqESkiCvR8KiuD88+HmhoYMACOOSYsTL1mTdyViUgR\naDXQzexmM1tqZvPStv3MzBab2UvR7cjclllkdtstrFd60UWhT32vveCBB9S3LiIdkkkL/VZgTDPb\nr3b3kdFtRnbLKgGdO4cLkR5/PAT5+PHwla/AI4/EXZmIFKhWA93dnwY+zUMtpemQQ2D+/DBufelS\nGDs2jIoREWmjjvShn2Nmr0RdMn1a2snMJplZjZnV1NbWduDjilhFRRi3/sYbsM8+cNllYRENEZE2\naG+gTwZ2AEYCS4Dft7Sju09x92p3r66qqmrnx5WITp3C2qULF8JNN8VdjYgUmHYFurt/7O4b3b0B\nuAnYN7tllbCvfS3MAfOrX2n0i4i0SbsC3cwGpj09FpjX0r7SRmahlf7xx3DddZu/3tAAZ50FDz+c\n/9pEJNEyGbZ4B/AvYBczW2RmpwFXmNlcM3sF+CpwQY7rLC0HHBAWz7jiCvjss8avTZsGkyfD71vs\n5RKREmWex7HP1dXVXlNTk7fPK2gvvxzGp59wAtx5Z2i5L18Ou+wCtbXhh9Rly6BHj7grFZEcM7M5\n7l7d2n66UjSp9twTfvMbuPvu0J8O8LOfhTD//e+hvh4eeyzWEkUkWRToSXbxxXDyyfDTn4ZQ/8Mf\nwuRe55wDvXrB9OlxVygiCVIRdwGyBWZhaoA334Sf/AT69g3BXlkJhx8elrtzD/uJSMlTCz3punSB\n+++HUaPghhtg663D9iOPhCVLwpJ3IiKohV4YBgyAZ55pvG3s2HA/Y0b48VRESp5a6IWqf3+orlY/\nuoh8QYFeyI46Cp57Dj75JO5KRCQBFOiF7Mgjw4+it94aZmx85x3YuDHuqkQkJgr0QlZdDQMHhoUy\nRoyAHXYIrXYtlCFSkvSjaCErK4Mnn4RXXw3T7dbUhIuObrsNvv3tuKsTkTzTpf/FpKEhDG98+21Y\nsCCMWxeRgqdL/0tRWVmYuGvZMvjRj+KuRkTyTIFebEaOhHPPDVeYPvdc3NWISB4p0IvRL34Rfiz9\nzndg5cq4qxGRPFGgF6MePcIPo2++CSeeqKGMIiVCgV6sRo8OKx5Nn67+dJESoWGLxeyss2DevLDy\n0a67wimnxF2RiOSQWujF7tprQ2v9O98J86jX1cVdkYjkiAK92FVWhm6XCy4I0+/uvXe4AElEio4C\nvRR06QJXXRWWrFuzJlx8dPvtcVclIlmmQC8lhx4aFp8+4ACYMCGsUap5X0SKhgK91PTtCzNnwsSJ\n8POfh2Bfvz7uqkQkCzTKpRR16gS33AI77ww//jG8/35Y5q5fv7grE5EOUAu9VJmF8el33gkvvAD7\n7w9vvBF3VSLSAQr0UvfNb8ITT8CKFeHH0mXL4q5IRNpJgS6hdT5zZljK7sYb465GRNpJgS7BXnvB\n4YfD9dfrR1KRAqVAl00uvBA++gjuuivuSkSkHRTossnXvgZf/nJYxk7j00UKjgJdNjGD738fXnkF\nHn887mokidzhd7+DRYvirkSaoUCXxk46Cfr3D1MFiDT18cdw8cVw991xVyLNUKBLY507w9lnw4wZ\nMH9+3NVI0qRm61y+PN46pFmtBrqZ3WxmS81sXtq2vmY2y8zejO775LZMyaszzwzBfv31cVciSaNA\nT7RMWui3AmOabLsUmO3uOwGzo+dSLKqqwtJ106bpf1xpTIGeaK0Gurs/DXzaZPN4YFr0eBrw9SzX\nJXE799zwP+/UqXFXIkmiQE+09vah93f3JdHjj4D+WapHkmLkSDjoIPjDH7TItGyiQE+0Dv8o6u4O\ntDho2cwmmVmNmdXU1tZ29OMkn849F957Dx58MO5KJCkU6InW3kD/2MwGAkT3S1va0d2nuHu1u1dX\nVVW18+MkFuPHw9ChcN11cVciSZEK9M8+i7cOaVZ7A/0BYGL0eCLwj+yUI4lSURGGMD7xBDz/fNzV\nSBKsXRvu1UJPpEyGLd4B/AvYxcwWmdlpwOXA18zsTeCw6LkUozPPhMGDYdIkqK+PuxqJW6qFvno1\nbNgQby2ymUxGufy3uw9090p3H+LuU919mbsf6u47ufth7t50FIwUi5494YYbwnQAV14ZdzUSt1Sg\nQ5hDXxJFV4pK68aPh+OPD2uQalWj0pYe6Op2SRytKSqZuf56eOyx0PUyezaUl2967fPP4amnwjwf\na9eG58ceC4MGxVev5IYCPdEU6JKZAQPCLHvf/W7oUx83Dg49FP75z7AuadOl6155RasfFSMFeqIp\n0CVzp50GffuGBTDuuANuugm6dAldMhMmwC67wFZbwfe+B488EqZaNYu7askmBXqiKdAlc2Zw3HHh\ntn49zJkTFsTo1avxfkcfHS5GWrAAdt01nlolN+rqoEcPWLVKgZ5A+lFU2qdzZzjggM3DHOCII8L9\nzJn5rUlyb+3aTb+NKNATR4Eu2bfddjB8eOh2keJSVwfbbANlZQr0BFKgS26MGRNGvqSuLJTiUFcH\n3bqFv8wU6ImjQJfcOOIIWLcOnn467kokm+rqwg/fvXsr0BNIgS65cfDBYQSM+tGLiwI90RTokhtd\nu4b51NP70d2hoSG+mqTjFOiJpkCX3BkzJiw0/dZbYT6YQYPglFPirko6QoGeaAp0yZ0x0VK0I0fC\nOeeEkRG33QYvvhhvXdJ+a9cq0BNMgS65M3x4CPOddoKHH4bXXoM+feCyy+KuTNqjvj5MmatATyxd\nKSq5YxZa4+mX/19yCVx6KTz7LIwaFV9t0napy/67dg2BnpoTvUIxkhRqoUtuNZ3L5ZxzoH//0Er3\nFpeilSRKBXqqhQ6aEz1hFOiSX926wY9/DE8+GabhlcLRXKCr2yVRFOiSf5MmwZAhcPXVcVcibaFA\nTzwFuuRf586w//5hOKMUjvRA79MnPFagJ4oCXeIxeDAsXqx+9EKiFnriKdAlHoMHw5o1sHJl3JVI\nplITrSnQE0uBLvEYMiTcL1oUbx2SObXQE0+BLvEYPDjcL14cbx2SufRx6N27a070BFKgSzwU6IUn\nvYVupqtFE0iBLvFILWOmLpfCkR7oEAL9s8/iq0c2o0CXeHTpAv36qYVeSJoLdLXQE0WBLvFJDV2U\nwlBXF+ZtqawMzxXoiaNAl/gMGaIul0KSmjo3RYGeOAp0iY9a6IUltbhFigI9cRToEp/Bg6G2Ftav\nj7sSyYQCPfEU6BKf1MVFH34Ybx2Smbq6MAY9pXfvcLVvfX18NUkjCnSJj8aiF5bmWuigOdETRIEu\n8VGgF5aWAl3dLonRobWjzOw9YBWwEdjg7tXZKEpKhOZzKSx1dVBVtem5Aj1xsrEY4Ffd/ZMs/HOk\n1PTqFVp8aqEXBrXQE09dLhIfMw1dLCTNjUMHBXqCdDTQHXjUzOaY2aTmdjCzSWZWY2Y1tbW1Hfw4\nKTq6uKhwqIWeeB0N9APdfW9gLHC2mR3UdAd3n+Lu1e5eXZXe/yYCaqEXEgV64nUo0N19cXS/FLgf\n2DcbRUkJGTw4jENvaIi7EtkS983HoXfvDj17wuuvx1eXNNLuQDezbmbWI/UYOByYl63CpEQMGRIu\nTFF3XLLV18PGjY1b6GZw6KEwc6bWhk2IjrTQ+wPPmNnLwL+B6e7+SHbKkpKhseiFoenUuSlHHAEf\nfKBWekK0e9iiu78D7JnFWqQUpQf63nvHW4u0rKVAP/zwcD9zJgwfnt+aZDMatijx0sVFhWHt2nDf\nNNC33x522ikEusROgS7x6t8fysvV5ZJ0LbXQIXS7PPkkrFuX15Jkcwp0iVd5OQwYoEBPutYCfe1a\neOaZ/NYkm1GgS/x0cVHybSnQDzkkLEunbpfYKdAlfjvsAC+/DJ9/Hncl0pJUoKePQ0/p3h0OPFCB\nngAKdInfSSeFcej33x93JdKSLbXQIXS7zJ0LS5bkrybZjAJd4nfEEWG0xOTJcVciLckk0AEefTQ/\n9UizFOgSv/JyOOMMeOopeO21uKuR5rQW6HvsEX7cfvDB/NUkm1GgSzKceip06qRWelK1NA49pawM\njjsOZswI64xKLBTokgxVVXD88fDnP8Pq1XFXI01t6UfRlOOPD8H/8MP5qUk2o0CX5DjrLFi5Eu64\nI+5KpKm6ujA0sbKy5X0OOih8Md9zT/7qkkYU6JIcBxwAu+8O110XZvaT5Gg6F3pzystDt8v06Zta\n9JJXCnRJDjO47DKYNy+EuiRH07nQW3LCCaEP/RFNvBoHBbokywknwFFHhWB/7724q5GUTFroAAcf\nDP36wb335r4m2YwCXZLFDP73f8OoiTPP1MIJSZFpoFdUwLHHhuGLmqwr7xTokjxDh8JvfhMuJb/9\n9rirEQijVzIJdAijXVav1lQAMVCgSzJ973uw335h5Muf/6yWetwybaEDfPWr0LcvTJuW25pkMwp0\nSabycrjrrnAF4sSJYfTE0qVxV1W62hLolZXhi/j+++Gll3JblzSiQJfkGjo0TAfwu9+FKxB32w2e\nfTbuqkpTWwId4Pvfh1694H/+J3c1yWYU6JJs5eXwgx/AnDkhIEaPVr96HNoa6H36hPP2wAPwwgu5\nq0saUaBLYdhtN3juOdh/f5gwAX74Q3j7bfWt50um49DTnXcebL01/PSnualJNqNAl8Kx9dZhetZT\nT4XLL4cdd4RBg+Bb31IrMNfa2kIH6NEDLrkkXGSkrrK8UKBLYenUCaZOhVdfhT/+EQ47DB57DPbd\nN/xwOm+eWu3Z5t6+QAc4++ywEPjFF0NDQ/Zrk0Yq4i5ApF1GjAi3M86AVavgmmvgyivDyIru3cM6\npYMHh+FzvXqFPt299w598NtsE3f1haW+PoRxewJ9q63CNQWnngo33wynn579+uQL5nlszVRXV3tN\nTU3ePk9KzLJl4QfTd98Ni04vXgyffQYrVsCnn8L69WG/3XeHMWPgmGNCn3yF2jVbtHx5+EK8+mo4\n//y2v989LCQ9dy4sWKAv1HYwsznuXt3afvovWYrH1lvDuec2/9qGDfDiizB7duiiueaaMByyb1/Y\nZ5+wUPWXvhS6B7p3Dzf3MNHUmjXQs2do4Q8ZEqYnKCWtrVbUGjO48cZwTcGFF8Jtt2WvNmlEgS6l\noaIi9LPvu28YIbNyZbg0fcaM0O9+992hFd+aqqoQ/qm5wbt2Ddv69Qu3Xr2gd+/wg2BlZfjcyspN\nXxI9eoT3dO0athfCl8PTT4f79gY6wPDhcOml8Mtfwsknw+GHZ6c2aURdLiIpy5eHbpvVq8PNDLp1\nC7dPPglj4efMCd059fXhtmZNeK22dlOXTqbKyqBLl81vXbs2vz312lZbhfvOnTd9saS/1qVL+PG4\nsjLsk/7+Tp3CLbW9U6dNXyruoa+8vDw8X7sWLroIbrghtK5nzw5fWu21bl3o7nr7bfjKV2DcODjw\nwPCXVZ8+4YuwS5fC+JLLs0y7XBToItmQGgmyYkW4rVoVunk2bIDPP9/0JbF6dQjK1G39+pYfr1sX\nHq9b1/hxXV3bvzy2pFOnUGdqFEq/fjBwYPgrZuFCuOAC+PWvQ9h21KJFcMst4YKj5rKgrCz8JdOt\n26YvrvQvsdQXVurWuXPjW+oLK/2W+tJL/4upuW0VFc3fyss3vy/L7wBBBbpIMWtoCF8U9fXhPv3L\nYN26sC21PfWF0HR76rX16zeFmjt8/DF8+GH48rnkktx1j3z4Yfih9LPPwm358vAXT9Mvvrq6xo/T\njyd1HPX1uamxJWabh3zq1vR5atuUKeEvknZ9nH4UFSle6d01hWrQoHDLhoaGEOrpAZ/6skt/vGHD\npuf19Zuep/6aSj3euHHT840bwy39cfo+qcfp25veNmwIf3nkmAJdRApfWdmmbpcS1qGOIDMbY2av\nm9lbZnZptooSEZG2a3egm1k5cAMwFhgB/LeZjchWYSIi0jYdaaHvC7zl7u+4++fAncD47JQlIiJt\n1ZFAHwx8kPZ8UbStETObZGY1ZlZTW1vbgY8TEZEtyflgSnef4u7V7l5dVVWV648TESlZHQn0xcC2\nac+HRNtERCQGHQn0F4CdzGx7M+sEfAt4IDtliYhIW7V7HLq7bzCzc4CZQDlws7u/mrXKRESkTfJ6\n6b+Z1QIL2/n2fsAnWSynUJTicZfiMUNpHncpHjO0/bi3c/dWf4TMa6B3hJnVZDKXQbEpxeMuxWOG\n0jzuUjxmyN1xa01REZEioUAXESkShRToU+IuICaleNyleMxQmsddiscMOTrugulDFxGRLSukFrqI\niGyBAl1EpEgURKCXwrzrZratmT1hZq+Z2atmdl60va+ZzTKzN6P7PnHXmm1mVm5m/2dmD0XPtzez\n56PzfVd0JXJRMbPeZnavmS0ws/lmtn+xn2szuyD6b3uemd1hZl2K8Vyb2c1mttTM5qVta/bcWnBd\ndPyvmNneHfnsxAd6Cc27vgG40N1HAPsBZ0fHeSkw2913AmZHz4vNecD8tOe/Ba529x2Bz4DTYqkq\nt64FHnH34cCehOMv2nNtZoOBc4Fqd9+NcHX5tyjOc30rMKbJtpbO7Vhgp+g2CZjckQ9OfKBTIvOu\nu/sSd38xeryK8D/4YMKxTot2mwZ8PZ4Kc8PMhgBHAX+KnhswGrg32qUYj7kXcBAwFcDdP3f35RT5\nuSZMNdLVzCqArYAlFOG5dvengU+bbG7p3I4H/uzBc0BvMxvY3s8uhEDPaN71YmJmw4C9gOeB/u6+\nJHrpI6B/TGXlyjXAxUBD9HxrYLm7b4ieF+P53h6oBW6Jupr+ZGbdKOJz7e6LgSuB9wlBvgKYQ/Gf\n65SWzm1W860QAr2kmFl34D7gfHdfmf6ahzGmRTPO1MyOBpa6+5y4a8mzCmBvYLK77wWsoUn3ShGe\n6z6E1uj2wCCgG5t3S5SEXJ7bQgj0kpl33cwqCWF+u7v/Ldr8cepPsOh+aVz15cAoYJyZvUfoShtN\n6FvuHf1ZDsV5vhcBi9z9+ej5vYSAL+ZzfRjwrrvXuns98DfC+S/2c53S0rnNar4VQqCXxLzrUd/x\nVGC+u1+V9tIDwMTo8UTgH/lqCYkaAAAA9ElEQVSuLVfc/YfuPsTdhxHO6+PufhLwBHB8tFtRHTOA\nu38EfGBmu0SbDgVeo4jPNaGrZT8z2yr6bz11zEV9rtO0dG4fAL4djXbZD1iR1jXTdu6e+BtwJPAG\n8Dbw47jrydExHkj4M+wV4KXodiShT3k28CbwGNA37lpzdPyHAA9Fj78E/Bt4C7gH6Bx3fTk43pFA\nTXS+/w70KfZzDfwcWADMA24DOhfjuQbuIPxOUE/4a+y0ls4tYIRRfG8DcwmjgNr92br0X0SkSBRC\nl4uIiGRAgS4iUiQU6CIiRUKBLiJSJBToIiJFQoEuIlIkFOgiIkXi/wM+2JhXmjL6WgAAAABJRU5E\nrkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "de5IzlVP6N0Y",
        "colab_type": "text"
      },
      "source": [
        "##Lets generate more paragraphs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TKQARfmZ6TeT",
        "colab_type": "code",
        "outputId": "8f9171d2-6565-4b3d-e09d-687ca721feb6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "print(generate(\"Machines can often act and react like\", num_chars = 500))"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Machines can often act and react like humans, th the problem is embedded. Intelligence—at is an AI system that past categorized as either weak net faniour is ass keduction and intensing (ust use that the term artificial intelligence must haviour is ascribed to intelligence and the ability to reason, discover mayike of ehamples, thereby discovering and comple computer program for solving post comple experience. Since the deveronely appeer entollecial purpose and general purpose.. A particular task or situation and numerical rigs. Ro\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "caM4rJ-w6TbN",
        "colab_type": "code",
        "outputId": "25266dae-e133-4c23-b217-1096b7e870f3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "print(generate(\"There are a number of different forms of learning\", num_chars = 500))"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "There are a number of different forms of learning as applicable object the guntral purpose and general purpose. many of the porentelligence as applicable to a wide various spocatide the pose the learn fious basine learning as applications in thais andurion diffinuaring consion involves applying past experience. Since the deverone human experts and procedures—knowniatares of the digital computer or computer science often regres an dafferenc of the digital computer or computer science often regres an daffearing closs find ssscess and search engi\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H4eFTYaU7IEC",
        "colab_type": "code",
        "outputId": "1d7aee44-be74-4a79-d620-1aff0f06ce91",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "print(generate(\"Problem solving, particularly in artificial intelligence, may be characterized as\", num_chars = 250))"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Problem solving, particularly in artificial intelligence, may be characterized as a systematic search the processes characterize human intelligence and the simulation of human intelligence as applicable or entollec on heware durpose terth a fowlate dusp behariding as applied to the project on a form that letrua ment ad and cimple\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}