geoLEARN/Submission_3_RF_FE.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Facies classification using Random Forest\n",
    "\n",
    "\n",
    "### Contest entry by <a href=\\\"https://geolern.github.io/index.html#\\\">geoLEARN</a>:\n",
    "\n",
    "\n",
    "##### <a href=\\\"https://github.com/mablou\\\">Martin Blouin</a>,  <a href=\\\"https://github.com/lperozzi\\\">Lorenzo Perozzi</a> and <a href=\\\"https://github.com/Antoine-Cate\\\">Antoine Caté</a> <br> \n",
    "in collaboration with <a href=\\\"http://ete.inrs.ca/erwan-gloaguen\\\">Erwan Gloaguen</a>\n",
    "\n",
    "\n",
    "####  [Original contest notebook](../Facies_classification.ipynb) by Brendon Hall, [Enthought](https://www.enthought.com/)\n",
    "\n",
    "In this notebook we will train a machine learning algorithm to predict facies from well log data.  The dataset comes from a class exercise from The University of Kansas on [Neural Networks and Fuzzy Systems](http://www.people.ku.edu/~gbohling/EECS833/).  This exercise is based on a consortium project to use machine learning techniques to create a reservoir model of the largest gas fields in North America, the Hugoton and Panoma Fields. For more info on the origin of the data, see [Bohling and Dubois (2003)](http://www.kgs.ku.edu/PRS/publication/2003/ofr2003-50.pdf) and [Dubois et al. (2007)](http://dx.doi.org/10.1016/j.cageo.2006.08.011).\n",
    "\n",
    "The dataset consists of log data from nine wells that have been labeled with a facies type based on observation of core. We will use this log data to train a Random Forest model to classify facies types. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring the dataset\n",
    "\n",
    "First, we import and examine the dataset used to train the classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Facies</th>\n",
       "      <th>Formation</th>\n",
       "      <th>Well Name</th>\n",
       "      <th>Depth</th>\n",
       "      <th>GR</th>\n",
       "      <th>ILD_log10</th>\n",
       "      <th>DeltaPHI</th>\n",
       "      <th>PHIND</th>\n",
       "      <th>PE</th>\n",
       "      <th>NM_M</th>\n",
       "      <th>RELPOS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2793.0</td>\n",
       "      <td>77.45</td>\n",
       "      <td>0.664</td>\n",
       "      <td>9.9</td>\n",
       "      <td>11.915</td>\n",
       "      <td>4.6</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2793.5</td>\n",
       "      <td>78.26</td>\n",
       "      <td>0.661</td>\n",
       "      <td>14.2</td>\n",
       "      <td>12.565</td>\n",
       "      <td>4.1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2794.0</td>\n",
       "      <td>79.05</td>\n",
       "      <td>0.658</td>\n",
       "      <td>14.8</td>\n",
       "      <td>13.050</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1</td>\n",
       "      <td>0.957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2794.5</td>\n",
       "      <td>86.10</td>\n",
       "      <td>0.655</td>\n",
       "      <td>13.9</td>\n",
       "      <td>13.115</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1</td>\n",
       "      <td>0.936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2795.0</td>\n",
       "      <td>74.58</td>\n",
       "      <td>0.647</td>\n",
       "      <td>13.5</td>\n",
       "      <td>13.300</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1</td>\n",
       "      <td>0.915</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Facies Formation  Well Name   Depth     GR  ILD_log10  DeltaPHI   PHIND  \\\n",
       "0       3     A1 SH  SHRIMPLIN  2793.0  77.45      0.664       9.9  11.915   \n",
       "1       3     A1 SH  SHRIMPLIN  2793.5  78.26      0.661      14.2  12.565   \n",
       "2       3     A1 SH  SHRIMPLIN  2794.0  79.05      0.658      14.8  13.050   \n",
       "3       3     A1 SH  SHRIMPLIN  2794.5  86.10      0.655      13.9  13.115   \n",
       "4       3     A1 SH  SHRIMPLIN  2795.0  74.58      0.647      13.5  13.300   \n",
       "\n",
       "    PE  NM_M  RELPOS  \n",
       "0  4.6     1   1.000  \n",
       "1  4.1     1   0.979  \n",
       "2  3.6     1   0.957  \n",
       "3  3.5     1   0.936  \n",
       "4  3.4     1   0.915  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "###### Importing all used packages\n",
    "%matplotlib inline\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.colors as colors\n",
    "from mpl_toolkits.axes_grid1 import make_axes_locatable\n",
    "import seaborn as sns\n",
    "\n",
    "from pandas import set_option\n",
    "# set_option(\"display.max_rows\", 10)\n",
    "pd.options.mode.chained_assignment = None\n",
    "\n",
    "###### Import packages needed for the make_vars functions\n",
    "import Feature_Engineering as FE\n",
    "\n",
    "##### import stuff from scikit learn\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import KFold, cross_val_score,LeavePGroupsOut, LeaveOneGroupOut, cross_val_predict\n",
    "from sklearn.metrics import confusion_matrix, make_scorer, f1_score, accuracy_score, recall_score, precision_score\n",
    "\n",
    "filename = '../facies_vectors.csv'\n",
    "training_data = pd.read_csv(filename)\n",
    "training_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Facies</th>\n",
       "      <th>Depth</th>\n",
       "      <th>GR</th>\n",
       "      <th>ILD_log10</th>\n",
       "      <th>DeltaPHI</th>\n",
       "      <th>PHIND</th>\n",
       "      <th>PE</th>\n",
       "      <th>NM_M</th>\n",
       "      <th>RELPOS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4149.000000</td>\n",
       "      <td>4149.000000</td>\n",
       "      <td>4149.000000</td>\n",
       "      <td>4149.000000</td>\n",
       "      <td>4149.000000</td>\n",
       "      <td>4149.000000</td>\n",
       "      <td>3232.000000</td>\n",
       "      <td>4149.000000</td>\n",
       "      <td>4149.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>4.503254</td>\n",
       "      <td>2906.867438</td>\n",
       "      <td>64.933985</td>\n",
       "      <td>0.659566</td>\n",
       "      <td>4.402484</td>\n",
       "      <td>13.201066</td>\n",
       "      <td>3.725014</td>\n",
       "      <td>1.518438</td>\n",
       "      <td>0.521852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2.474324</td>\n",
       "      <td>133.300164</td>\n",
       "      <td>30.302530</td>\n",
       "      <td>0.252703</td>\n",
       "      <td>5.274947</td>\n",
       "      <td>7.132846</td>\n",
       "      <td>0.896152</td>\n",
       "      <td>0.499720</td>\n",
       "      <td>0.286644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>2573.500000</td>\n",
       "      <td>10.149000</td>\n",
       "      <td>-0.025949</td>\n",
       "      <td>-21.832000</td>\n",
       "      <td>0.550000</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>2821.500000</td>\n",
       "      <td>44.730000</td>\n",
       "      <td>0.498000</td>\n",
       "      <td>1.600000</td>\n",
       "      <td>8.500000</td>\n",
       "      <td>3.100000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.277000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>4.000000</td>\n",
       "      <td>2932.500000</td>\n",
       "      <td>64.990000</td>\n",
       "      <td>0.639000</td>\n",
       "      <td>4.300000</td>\n",
       "      <td>12.020000</td>\n",
       "      <td>3.551500</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.528000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>3007.000000</td>\n",
       "      <td>79.438000</td>\n",
       "      <td>0.822000</td>\n",
       "      <td>7.500000</td>\n",
       "      <td>16.050000</td>\n",
       "      <td>4.300000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.769000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>9.000000</td>\n",
       "      <td>3138.000000</td>\n",
       "      <td>361.150000</td>\n",
       "      <td>1.800000</td>\n",
       "      <td>19.312000</td>\n",
       "      <td>84.400000</td>\n",
       "      <td>8.094000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Facies        Depth           GR    ILD_log10     DeltaPHI  \\\n",
       "count  4149.000000  4149.000000  4149.000000  4149.000000  4149.000000   \n",
       "mean      4.503254  2906.867438    64.933985     0.659566     4.402484   \n",
       "std       2.474324   133.300164    30.302530     0.252703     5.274947   \n",
       "min       1.000000  2573.500000    10.149000    -0.025949   -21.832000   \n",
       "25%       2.000000  2821.500000    44.730000     0.498000     1.600000   \n",
       "50%       4.000000  2932.500000    64.990000     0.639000     4.300000   \n",
       "75%       6.000000  3007.000000    79.438000     0.822000     7.500000   \n",
       "max       9.000000  3138.000000   361.150000     1.800000    19.312000   \n",
       "\n",
       "             PHIND           PE         NM_M       RELPOS  \n",
       "count  4149.000000  3232.000000  4149.000000  4149.000000  \n",
       "mean     13.201066     3.725014     1.518438     0.521852  \n",
       "std       7.132846     0.896152     0.499720     0.286644  \n",
       "min       0.550000     0.200000     1.000000     0.000000  \n",
       "25%       8.500000     3.100000     1.000000     0.277000  \n",
       "50%      12.020000     3.551500     2.000000     0.528000  \n",
       "75%      16.050000     4.300000     2.000000     0.769000  \n",
       "max      84.400000     8.094000     2.000000     1.000000  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A complete description of the dataset is given in the  [Original contest notebook](../Facies_classification.ipynb) by Brendon Hall, [Enthought](https://www.enthought.com/). A total of four measured rock properties and two interpreted geological properties are given as raw predictor variables for the prediction of the \"Facies\" class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature engineering\n",
    "As stated in our [previous submission](Submission_increased_variance.ipynb), we believe that feature engineering has a high potential for increasing classification success. A strategy for building new variables is explained below.\n",
    "\n",
    "The dataset is distributed along a series of drillholes intersecting a stratigraphic sequence. Sedimentary facies tend to be deposited in sequences that reflect the evolution of the paleo-environment (variations in water depth, water temperature, biological activity, currents strenght, detrital input, ...). Each facies represents a specific depositional environment and is in contact with facies that represent a progressive transition to an other environment.\n",
    "Thus, there is a relationship between neighbouring samples, and the distribution of the data along drillholes can be as important as data values for predicting facies.\n",
    "\n",
    "A series of new variables (features) are calculated and tested below to help represent the relationship of neighbouring samples and the overall texture of the data along drillholes. These variables are:\n",
    "\n",
    "- detail and approximation coeficients at various levels of two [wavelet transforms](https://en.wikipedia.org/wiki/Discrete_wavelet_transform) (using two types of [Daubechies](https://en.wikipedia.org/wiki/Daubechies_wavelet) wavelets);\n",
    "- measures of the local entropy with variable observation windows;\n",
    "- measures of the local gradient with variable observation windows;\n",
    "- rolling statistical calculations (i.e., mean, standard deviation, min and max) with variable observation windows;\n",
    "- ratios between marine and non-marine lithofacies with different observation windows;\n",
    "- distances from the nearest marine or non-marine occurence uphole and downhole.\n",
    "\n",
    "Functions used to build these variables are located in the [Feature Engineering](Feature_Engineering.py) python script.\n",
    "\n",
    "All the data exploration work related to the conception and study of these variables is not presented here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "##### cD From wavelet db1\n",
    "dwt_db1_cD_df = FE.make_dwt_vars_cD(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                       levels=[1, 2, 3, 4], wavelet='db1')\n",
    "\n",
    "##### cA From wavelet db1\n",
    "dwt_db1_cA_df = FE.make_dwt_vars_cA(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                       levels=[1, 2, 3, 4], wavelet='db1')\n",
    "\n",
    "##### cD From wavelet db3\n",
    "dwt_db3_cD_df = FE.make_dwt_vars_cD(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                       levels=[1, 2, 3, 4], wavelet='db3')\n",
    "\n",
    "##### cA From wavelet db3\n",
    "dwt_db3_cA_df = FE.make_dwt_vars_cA(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                       levels=[1, 2, 3, 4], wavelet='db3')\n",
    "\n",
    "##### From entropy\n",
    "entropy_df = FE.make_entropy_vars(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                               l_foots=[2, 3, 4, 5, 7, 10])\n",
    "\n",
    "###### From gradient\n",
    "gradient_df = FE.make_gradient_vars(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                 dx_list=[2, 3, 4, 5, 6, 10, 20])\n",
    "\n",
    "##### From rolling average\n",
    "moving_av_df = FE.make_moving_av_vars(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                   windows=[1, 2, 5, 10, 20])\n",
    "\n",
    "##### From rolling standard deviation\n",
    "moving_std_df = FE.make_moving_std_vars(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                     windows=[3 , 4, 5, 7, 10, 15, 20])\n",
    "\n",
    "##### From rolling max\n",
    "moving_max_df = FE.make_moving_max_vars(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                     windows=[3, 4, 5, 7, 10, 15, 20])\n",
    "\n",
    "##### From rolling min\n",
    "moving_min_df = FE.make_moving_min_vars(wells_df=training_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                     windows=[3 , 4, 5, 7, 10, 15, 20])\n",
    "\n",
    "###### From rolling NM/M ratio\n",
    "rolling_marine_ratio_df = FE.make_rolling_marine_ratio_vars(wells_df=training_data, windows=[5, 10, 15, 20, 30, 50, 75, 100, 200])\n",
    "\n",
    "###### From distance to NM and M, up and down\n",
    "dist_M_up_df = FE.make_distance_to_M_up_vars(wells_df=training_data)\n",
    "dist_M_down_df = FE.make_distance_to_M_down_vars(wells_df=training_data)\n",
    "dist_NM_up_df = FE.make_distance_to_NM_up_vars(wells_df=training_data)\n",
    "dist_NM_down_df = FE.make_distance_to_NM_down_vars(wells_df=training_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4149, 299)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Facies</th>\n",
       "      <th>Formation</th>\n",
       "      <th>Well Name</th>\n",
       "      <th>Depth</th>\n",
       "      <th>GR</th>\n",
       "      <th>ILD_log10</th>\n",
       "      <th>DeltaPHI</th>\n",
       "      <th>PHIND</th>\n",
       "      <th>PE</th>\n",
       "      <th>NM_M</th>\n",
       "      <th>...</th>\n",
       "      <th>Marine_ratio_20_centered</th>\n",
       "      <th>Marine_ratio_30_centered</th>\n",
       "      <th>Marine_ratio_50_centered</th>\n",
       "      <th>Marine_ratio_75_centered</th>\n",
       "      <th>Marine_ratio_100_centered</th>\n",
       "      <th>Marine_ratio_200_centered</th>\n",
       "      <th>dist_M_up</th>\n",
       "      <th>dist_M_down</th>\n",
       "      <th>dist_NM_up</th>\n",
       "      <th>dist_NM_down</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2793.0</td>\n",
       "      <td>77.45</td>\n",
       "      <td>0.664</td>\n",
       "      <td>9.9</td>\n",
       "      <td>11.915</td>\n",
       "      <td>4.6</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.140000</td>\n",
       "      <td>1.510000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>21.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2793.5</td>\n",
       "      <td>78.26</td>\n",
       "      <td>0.661</td>\n",
       "      <td>14.2</td>\n",
       "      <td>12.565</td>\n",
       "      <td>4.1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.156863</td>\n",
       "      <td>1.504950</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2794.0</td>\n",
       "      <td>79.05</td>\n",
       "      <td>0.658</td>\n",
       "      <td>14.8</td>\n",
       "      <td>13.050</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.173077</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>20.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2794.5</td>\n",
       "      <td>86.10</td>\n",
       "      <td>0.655</td>\n",
       "      <td>13.9</td>\n",
       "      <td>13.115</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.188679</td>\n",
       "      <td>1.495146</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>A1 SH</td>\n",
       "      <td>SHRIMPLIN</td>\n",
       "      <td>2795.0</td>\n",
       "      <td>74.58</td>\n",
       "      <td>0.647</td>\n",
       "      <td>13.5</td>\n",
       "      <td>13.300</td>\n",
       "      <td>3.4</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.203704</td>\n",
       "      <td>1.490385</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>19.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 299 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Facies Formation  Well Name   Depth     GR  ILD_log10  DeltaPHI   PHIND  \\\n",
       "0       3     A1 SH  SHRIMPLIN  2793.0  77.45      0.664       9.9  11.915   \n",
       "1       3     A1 SH  SHRIMPLIN  2793.5  78.26      0.661      14.2  12.565   \n",
       "2       3     A1 SH  SHRIMPLIN  2794.0  79.05      0.658      14.8  13.050   \n",
       "3       3     A1 SH  SHRIMPLIN  2794.5  86.10      0.655      13.9  13.115   \n",
       "4       3     A1 SH  SHRIMPLIN  2795.0  74.58      0.647      13.5  13.300   \n",
       "\n",
       "    PE  NM_M      ...       Marine_ratio_20_centered  \\\n",
       "0  4.6     1      ...                            1.0   \n",
       "1  4.1     1      ...                            1.0   \n",
       "2  3.6     1      ...                            1.0   \n",
       "3  3.5     1      ...                            1.0   \n",
       "4  3.4     1      ...                            1.0   \n",
       "\n",
       "   Marine_ratio_30_centered  Marine_ratio_50_centered  \\\n",
       "0                       1.0                       1.0   \n",
       "1                       1.0                       1.0   \n",
       "2                       1.0                       1.0   \n",
       "3                       1.0                       1.0   \n",
       "4                       1.0                       1.0   \n",
       "\n",
       "   Marine_ratio_75_centered  Marine_ratio_100_centered  \\\n",
       "0                       1.0                   1.140000   \n",
       "1                       1.0                   1.156863   \n",
       "2                       1.0                   1.173077   \n",
       "3                       1.0                   1.188679   \n",
       "4                       1.0                   1.203704   \n",
       "\n",
       "   Marine_ratio_200_centered  dist_M_up  dist_M_down  dist_NM_up  dist_NM_down  \n",
       "0                   1.510000       -1.0         21.5         0.0           0.0  \n",
       "1                   1.504950       -1.0         21.0         0.0           0.0  \n",
       "2                   1.500000       -1.0         20.5         0.0           0.0  \n",
       "3                   1.495146       -1.0         20.0         0.0           0.0  \n",
       "4                   1.490385       -1.0         19.5         0.0           0.0  \n",
       "\n",
       "[5 rows x 299 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list_df_var = [dwt_db1_cD_df, dwt_db1_cA_df, dwt_db3_cD_df, dwt_db3_cA_df,\n",
    "               entropy_df, gradient_df, moving_av_df, moving_std_df, moving_max_df, moving_min_df,\n",
    "              rolling_marine_ratio_df, dist_M_up_df, dist_M_down_df, dist_NM_up_df, dist_NM_down_df]\n",
    "combined_df = training_data\n",
    "for var_df in list_df_var:\n",
    "    temp_df = var_df\n",
    "    combined_df = pd.concat([combined_df,temp_df],axis=1)\n",
    "combined_df.replace(to_replace=np.nan, value='-1', inplace=True)\n",
    "print (combined_df.shape)\n",
    "combined_df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building a prediction model from these variables\n",
    "\n",
    "A Random Forest model is built here to test the effect of these new variables on the prediction power. Algorithm parameters have been tuned so as to take into account the non-stationarity of the training and testing sets using the LeaveOneGroupOut cross-validation strategy. The size of individual tree leafs and nodes has been increased to the maximum possible without significantly increasing the variance so as to reduce the bias of the prediction.\n",
    "Box plot for a series of scores obtained through cross validation are presented below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Create predictor and target arrays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = combined_df.iloc[:, 4:]\n",
    "y = combined_df['Facies']\n",
    "groups = combined_df['Well Name']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Estimation of validation scores from this tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "scoring_param = ['accuracy', 'recall_weighted', 'precision_weighted','f1_weighted']\n",
    "scores = []\n",
    "\n",
    "Cl = RandomForestClassifier(n_estimators=100, max_features=0.1, min_samples_leaf=25,\n",
    "                            min_samples_split=50, class_weight='balanced', random_state=42, n_jobs=-1)\n",
    "\n",
    "lpgo = LeavePGroupsOut(n_groups=2)\n",
    "\n",
    "for scoring in scoring_param:\n",
    "    \n",
    "    cv=lpgo.split(X, y, groups)\n",
    "    validated = cross_val_score(Cl, X, y, scoring=scoring, cv=cv, n_jobs=-1)\n",
    "    scores.append(validated)\n",
    "    \n",
    "scores = np.array(scores)\n",
    "scores = np.swapaxes(scores, 0, 1)\n",
    "scores = pd.DataFrame(data=scores, columns=scoring_param)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAArkAAAIkCAYAAAAapjMpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3XlcVmX+//E3ynITabgiictkxU1i4J4jSq5tLjjZ5OiY\nlkSZOtaMqThOmE5uaZloJkpMozZZWlp9x29lppXllrn9jEpJFEdIVDThDhKu3x9+ucdbsISAg8fX\n8/HoMZzrbJ9zn8P45rqvc46XMcYIAAAAsJEaVhcAAAAAVDRCLgAAAGyHkAsAAADbIeQCAADAdgi5\nAAAAsB1CLgAAAGyHkAsAAADbIeQCAADAdgi5AAAAsB1CLmCxvXv3avz48erWrZsiIiLUq1cvPfXU\nU8rIyPBYzul0asGCBVVa24IFCxQWFuaePnv2rB599FFFRkaqY8eOSk9Pl9Pp1Jo1ayp0vxs2bNCE\nCRPc09u2bZPT6dT27dsrdD9Xk6ysLA0ZMkS33nqrOnfurPz8/Ard/s6dOxUXF1eh26wI3bt3V3x8\nvNVlWKq6nhugsnlbXQBwNVuxYoVmzJihjh07aty4cWrYsKEOHTqkpUuX6r333tM///lPhYaGWlbf\nfffdp65du7qn165dq40bN2rKlCm68cYbdf311+v1119XkyZNKnS/KSkp8vLyck+3bNlSr7/+ulq0\naFGh+7ma/OMf/9Du3bs1d+5cNWzYUH5+fhW6/TfeeEMHDx6s0G2iYnBucLUi5AIW+eKLLzR9+nQN\nHTpUEydOdLe3b99ePXr00IABAzRp0iStXr3ashqDgoIUFBTknj516pS8vLw0aNAgd9utt95a6XUE\nBARUyX7sLCcnRw0bNtQdd9xhdSkAUCUYrgBYJDk5WbVr19YTTzxRYl7dunUVHx+vnj176scffyx1\n/a+//lpjxoxRp06dFB4erq5du+rvf/+7CgoK3Mts3rxZ999/v1q3bq0OHTroscceU1pamnv+kSNH\nNHLkSHXs2FGRkZEaNGiQNm3a5J6fmJgop9MpSRo6dKgWLFggY4ycTqfi4+N19OjREsMVvvvuO40e\nPVodO3ZUhw4d9Oijj3r0Ih09elTjx49Xly5dFB4ert/+9reaMGGCTp8+7d7P9u3btW3bNoWFhbl/\nvni4wt69exUbG6uOHTuqbdu2evTRR3XgwAH3/OJ1Pv/8c40YMUKRkZGKiorSnDlzZIy55Hkxxuj5\n559Xjx491KpVK/Xo0UPPPfeczp07517m7NmzmjZtmrp27arWrVtr4MCBHp9bUVGRVqxYob59+yoi\nIkLdunXT3LlzPc5NfHy8hg8frilTpqht27bq06ePjDEyxigpKUm9e/dWq1atdMcdd2j58uUeNf7S\nebtY9+7dtWbNGv3nP/9RWFiYe9jL8ePHFR8fr9tvv10RERG67777tGHDBo91i4fJ3HvvvYqIiNCL\nL75YYvvx8fF66623dPToUYWFhWnNmjWXHGIydOhQPfDAAx61JSYmavbs2ercubMiIiIUGxur9PR0\nj/V27NihoUOHuofKTJw4USdPnvRYJjU1VQ8++KBat26t7t2765133rnkZ1LszTfflNPp1Jdffqn+\n/fsrIiJC/fr103vvveex3KWu25ycHI9jmTFjhoYPH66IiAj97W9/c9f1S7+rTqdTr732muLj49Wu\nXTt17NhRzzzzjPLz8zVr1ix16tRJHTt21OTJkz3W+6XrpbRzI0kFBQWaPXu2br/9drVq1Ur9+vXT\nv//9b49jvtTxvPLKK7rrrrt06623qmvXrnr66ad19uzZX/ysgSpnAFji1ltvNU888cRlLx8aGmoS\nExONMcZ8//33pm3btmbEiBFm48aN5rPPPjMzZ840oaGhJikpyRhjzOHDh01ERISZNm2a2bp1q/ng\ngw/MnXfeaXr27GmMMaaoqMjceeedZvjw4ebjjz82n332mXnkkUdMy5YtzeHDh40xxiQmJhqn02mM\nMebAgQPmr3/9q3E6nWb37t3m8OHDJiMjw4SGhpq33nrLGGNMZmamadeunenTp49Zt26d2bhxoxk4\ncKCJiooyp0+fNi6Xy3Tr1s0MHDjQrF+/3mzdutW89NJLpmXLliYhIcG9nwEDBpgBAwaY3bt3m7Nn\nz5qtW7cap9Nptm3bZowx5vPPPzctW7Y0sbGxZsOGDWbdunWmf//+pm3btiYtLc0YY8zWrVtNaGio\n6dy5s3nxxRfNli1b3J/RypUrL/k5v/TSS6ZDhw7mrbfeMtu3bzdLly41t9xyi/uzLywsNPfdd5/p\n0KGDWbFihfnss8/MhAkTTMuWLc2OHTuMMcZMmjTJhIeHm8TERPPZZ5+ZpUuXmsjISDNixAj3fiZO\nnGhatmxpHnnkEfP555+bDRs2GGOM+dvf/mbCw8PNggULzObNm83zzz9vwsLCzIsvvnjZ5+1iX331\nlYmLizNRUVFm9+7dJjMz02RnZ5suXbqY3r17m7ffftts2rTJPP7448bpdJp33nnH47pr1aqVSUlJ\nMRs3bjQHDhwosf3Dhw97bP/kyZMlzlmxP/7xj2bo0KHu6W7dupl27dqZRx55xHz88cfmnXfeMR07\ndjT333+/e5lt27aZli1bmri4OLNx40azZs0a061bN9OnTx+Tn5/vce3dd999ZsOGDWbNmjWma9eu\npmXLlmbixImXPN9vvvmmCQ0NNZ06dTIvvvii+eSTT8zYsWON0+k0mzZtMsaYy7pui4+lZcuW5rnn\nnjObN282u3btuqzf1eLPuW3btiYhIcFs2bLFzJgxw4SGhpo777zTPPHEE2bz5s1mwYIFJjQ01CQn\nJ7vX+6XrpbRzY4wxI0aMMG3btjWvvPKK+fTTT01CQoIJDQ01a9asKXE8c+fOdR/Pu+++a8LDw83y\n5cvN9u3bzcqVK03r1q1/9jMGrELIBSxw4sQJExoaaubOnXvZ61wYcj/99FPzxz/+0eTl5Xks07dv\nXxMbG2uMMeZ//ud/jNPpNN9//717/p49e8zzzz9vcnNzzfHjx01oaKh599133fN/+OEHM3PmTHeQ\nuTDkljZ9ccidOXOmiYyMNCdOnHAvk5mZabp162Y2bdpkvvrqKzNkyBCTkZHhUfejjz5q7rrrLvf0\nxUGoOLAWB6aBAweaPn36mKKiIvcyZ86cMR06dDCPP/64xzrz58/32FePHj3Mo48+eolP+fw//g89\n9JBH2/Lly83bb79tjDFmw4YNJjQ01B1Kiw0aNMgsWLDAHDhwwISGhpolS5Z4zF+7dq0JDQ11B6eJ\nEycap9NpsrKy3Mt89913xul0mqVLl3qsO2/ePBMREWFycnIu67yVZuLEiaZ79+7u6dmzZ5tWrVqZ\nY8eOeSw3fPhwExUV5Z4ODQ0t8XlczvbLEnJ79OjhcS4XLFhgnE6nycnJMcYYc//995t+/fp5bOfQ\noUPmlltuMStWrDDGnL/2Wrdu7V7HGGN2795tQkNDLyvkLlq0yKM9JibGHbQv97rt1q2b6d27t8cy\nl/O7asz5z/nCYF9YWGgiIyNNz549TWFhobu9T58+ZtSoUcYYY9LS0n7xejGm5Ln59NNPTWhoqFm3\nbp3Hek8++aTp0qWLe3+lHc9TTz3lcczGGPPOO++Y5cuXG6C6YUwuYAFv7/O/eoWFheVav3Pnzurc\nubPOnTungwcPKj09Xd98841OnjypOnXqSJIiIiLk6+ure++9V3feeae6du2qDh06qFWrVpKka665\nRjfeeKMmT56sTz75RFFRUeratavHUw3KaufOnYqMjFTdunXdbUFBQR5fgS9fvlzGGKWnp+vQoUM6\nePCg0tLSLvuzcLlc2rdvn0aPHu1xc1qtWrXUvXt3ffzxxx7LR0REeEw3atRILpfrktvv2LGj5s6d\nqyFDhqh79+66/fbbNWTIEI9j9PHxUbdu3TzW+9e//uX+Xy8vL91zzz0e8++55x7Fx8dr27Zt7pv5\nAgMD1bBhQ/cyW7ZskSTdfvvtHp9Ht27dtGjRIu3YsUM9evSokPO2fft2tW7dWo0aNfJo79evnyZN\nmqSDBw+6b/Sr7JsfW7Vq5XEui2tyuVzy8/PTnj17FBsb6/GZNG7cWDfccIM+++wzDR48WDt37lTr\n1q113XXXuZe59dZbdf311//i/r28vBQTE+PR1qtXLy1YsEAFBQVyOp2Xfd0WD+8pdjm/q8Vat27t\n/rlGjRqqU6eOWrZsqRo1/juyMDAwUGfOnJF0+dfLxbZs2aIaNWooOjq6xHpvv/22vvnmG/dxXHw8\nHTt21MqVKzVgwAD17NlT0dHR6tOnzyU+WcBahFzAArVr11ZAQID+85//XHIZl8uln376SbVr1y4x\nzxijuXPn6tVXX5XL5VJwcLBatWolPz8/93jTxo0ba/ny5VqyZIlWrVqlZcuWqVatWho8eLAef/xx\nSeefYrBo0SK9//77Wrt2rWrWrKlevXpp6tSpqlWrVpmPKycnRyEhIT+7TEpKihYvXqzTp0+rXr16\nCg8Pl7+/v3744YefXa84BJ05c0bGGDVo0KDEMvXr13cHgOJ1/P39S2ynqKjokvt5+OGHFRAQoNWr\nV2vu3Ll69tlnddNNN2ny5Mnq2LGjcnJyFBgYeMn1i8cW169f36O9Zs2aqlOnjkd911xzjccyOTk5\nMsaUCMjFdX///feSKua8nT59utSnYhTXfeH5uLjOiuZwODymi0NdUVGRTp8+raKiIi1ZskRJSUke\ny3l5eblru9S1V9p1UpoL/9iQpHr16skYozNnzqh+/fqXfd1e/Fldzu9qsYCAgBJ1XXz9Xuj06dOX\ndb1cLCcnR0VFRR6h+uL1isPtxcdz9913S5JeffVVLVq0SImJiWrcuLHGjRunu+6665K1AlYg5AIW\niYqK0tatW1VQUCBfX98S81euXKnZs2dr9erVHs+qlaTFixfrlVde0bRp09SzZ09de+21ks4/8utC\nrVq10vz583Xu3Dl98cUXWrlypRYvXqywsDDdcccdatCggZ566ik99dRTSk1N1XvvvaekpCTVrVvX\nfZNJWdSqVUunTp0q0f7555+rSZMm+vLLLzVr1ixNmDBBAwYMcIfFxx9/XHv37v3ZbRcHgtq1a8vL\ny0vHjx8vsczx48dL9I6Vx+DBgzV48GCdPHlSH3/8sRYtWqQ//elP2rx5s2rVquVxs1Gxr776SsYY\nd09idna2goOD3fPPnTunU6dO/Wx9tWrVkpeXl/75z3+WGiyLt1cR5+26665TdnZ2ifbiYHRhb3x5\nFP9RcnFPZ15eXqlh7lKuvfZaeXl5afjw4aX2GBYH5Dp16ujEiRMl5pd2rkqTk5PjcczHjx9XzZo1\ndd111+mdd94p93V7ub+r5XG510tp6wUEBGjZsmWl3oTZrFmzn93v3Xffrbvvvltnz57V5s2btWTJ\nEj355JNq167dZf9RAVQFnq4AWOShhx7SqVOnNG/evBLzjh8/rpSUFN10000lAq50/ivzm266STEx\nMe5/NLOysvTNN9+4/9F65ZVX1L17d/3000/y9vZWx44dNXXqVBlj9J///Ee7du1S586dtW/fPknn\nv5YcO3asbr75Zh09erRcx9SuXTvt2rXLI1icOHFCDz/8sDZt2qSdO3fquuuu04MPPugOCrm5ufri\niy88/rGtWbNmiW0XhyZ/f3+Fh4frf//3fz3W+eGHH/TRRx+pXbt25aq92KBBg/TMM89IOh/0YmJi\nNGTIEJ05c0Znz55Vu3btdO7cOX3yySce602cOFGLFy9Whw4dZIzRu+++6zH/3XffVVFR0c/W1759\ne0nSyZMn1bJlS/d/2dnZmjdvnnJycirsvLVv315ffvmljh075tH+9ttvq379+mratOllb0uSx1fq\n0vlwaoxRZmamu+306dNlfl5rQECAbrnlFn333Xcen8mNN96o+fPna9u2bZKkTp066csvv/TovTxw\n4ICOHDnyi/swxmj9+vUebe+//77atm0rHx+fy75uS3M5v6vldTnXi1Ty3HTo0EF5eXkqKiryWC81\nNVWJiYkeTxK52BNPPKHRo0dLOn+O77jjDo0cOVKFhYWX7DkGrEJPLmCRiIgIjR07Vi+88IIOHjyo\nmJgY1alTR998841efvllFRQUlBqApfNjDRctWqSkpCS1bt1ahw4dUlJSkn766Sfl5eVJkm677TbN\nnTtXo0aN0pAhQ1SzZk299tpr8vPzU/fu3RUcHCyHw6Hx48dr9OjRql+/vjZv3qzU1FQNGzasXMc0\nfPhwrVmzRg899JAeffRReXt766WXXtL111+vfv36af369Xrttdc0a9YsdevWTVlZWXr55Zd14sQJ\nj2EZtWvX1q5du7RlyxbdcsstkuQRCP785z/r4YcfVmxsrIYMGaKCggL38Y8aNcq9XHlCRIcOHfTy\nyy+rfv36at26tTIzM5WSkqIOHTooMDDQ/bitiRMnauzYsWrSpInWrFmj7777Ts8884xatGihAQMG\naP78+XK5XGrfvr3279+vBQsW6LbbblOXLl0uue+bb75Zffv21d/+9jdlZGQoPDxcaWlpmjdvnpo0\naaLf/OY3+umnn+Tv7/+rz9uDDz6ot99+W8OHD9eoUaMUGBiot956S9u2bdOMGTPK/LnVrl1bJ06c\n0Mcff6ywsDCFhoYqODhYL774ojvcJSUllWvow5///Gc98sgjGjdunPr27avCwkK9/PLL2rt3r/t8\nDxs2TKtXr9aIESM0ZswYnTt3TvPmzSv1W5LSzJ49Wz/++KN+85vf6PXXX1daWpr++c9/Sjr/+3Y5\n121pLud3tbwu53qRSp6b6OhotWvXTiNHjtRjjz2mFi1aaPfu3UpMTFR0dPTPDse57bbbNGXKFM2a\nNUvR0dE6ffq0FixYoObNm5cYvwtYjZALWOjRRx9Vy5Yt3W8+O336tBo1aqTu3bvrkUce8XgRg5eX\nl7s3My4uTjk5OVq2bJkWLVqk4OBg9e/fXzVq1NDixYt19uxZhYaG6qWXXtLChQs1btw4nTt3TuHh\n4Xr55ZfdX0empKRozpw5mj59us6cOaNmzZpp6tSpHjfhXHhD0C9NN2rUSP/61780e/ZsxcfHy8fH\nR7fddpvmzZunWrVqacCAATp69KhWr16tf/3rXwoKCtLtt9+uwYMH66mnnlJaWppuuOEGDRkyRPv2\n7VNcXJxmzJihBg0aeOynU6dOSklJ0fz58/WXv/xFvr6+at++vZ599lmPt6JdXOsvtUvnv4L29fXV\nm2++qRdffNF9Q9tf/vIXSed7xZYuXaq5c+dq/vz5ysvLk9Pp1Msvv6zw8HBJ0vTp09W8eXOtXr1a\nS5YsUVBQkIYPH66RI0f+Yh0zZ87U4sWLtXLlSs2bN0/169dXnz59NHbsWHl5ecnX11cvv/zyL563\nXzru+vXr67XXXtPcuXP1zDPPuG+wWrRokW6//XaPdX7u8yr2u9/9Th9//LFGjRqlP/3pT3r44YeV\nmJio6dOn6y9/+Yvq1aun4cOHKy0tzeNZzZez/c6dO2vp0qVauHChHn/8cfn4+Khly5b6xz/+4X5J\nSGBgoF599VVNnz5d8fHxuuaaaxQbG6t169b9Yu1eXl6aMmWKXnrpJWVkZCgsLEz/+Mc/1KZNG0m6\n7Ou2tGO5nN/V4iEZpf1ulfbZXNj2S9fLpc7NkiVL9MILLygpKUknTpxQUFCQHnroIT322GM/u//7\n779f586d02uvveb+o7lz584aN25cqd/AAFbyMr/2+5IKUFBQoClTpuiDDz6Qw+HQQw89pAcffLDE\ncsUPib/Yvffe6/56sV27dsrNzXX34Hh5eWnnzp0/O3gfAHB1euuttzRp0iR9+OGHl/UkBgBXjmrR\nkztr1izt379fy5YtU0ZGhiZMmKDGjRurd+/eHsstXLhQP/30k3t6165deuKJJ9yP98nKylJubq7W\nr1/vcbcuARcAAODqYnnIdblcWrVqlZKTk+V0OuV0OhUbG6vly5eXCLkXjn0qKirS888/r4cfftg9\nZi8tLU0NGjRQ48aNq/QYAAAAUL1Y/nSF1NRUFRYWKjIy0t3Wtm1b7dmz52fXW716tU6fPq3Y2Fh3\n24EDB9S8efPKKhUAYDMDBgzQV199xVAFwIYsD7nHjx9XYGCg+w1Q0vmHcOfn55f6vM1iS5cu1fDh\nwz2GIhw8eFAul0tDhw5VVFSU4uLidOjQocosHwAAANVQtRiucPEjXoqnCwoKSl1ny5Yt+v7770s8\nTDstLU1nzpzRX/7yFwUEBGjJkiUaPny4/v3vf1/WY2vatWun/Pz8Em++AQAAQPXw/fffy8/PTzt2\n7PjZ5SwPuX5+fiXCbPH0pW4Ye//999WlS5cSzydMTk7WuXPn3OvNmTNH0dHR+uijj0p97eHFCgoK\nSrydBwAAANVHYWHhJTtCL2R5yA0KCnK/R7v4rSzZ2dlyOByXfMj2J598ojFjxpRo9/HxkY+Pj3va\n19dXISEhysrKuqxail9H+OGHH5b1MAAAAFAFevTocVnLWT4mNywsTN7e3tq1a5e7bceOHe6Hql/s\n1KlTOnLkiPsh3Rfq1auX1qxZ457Oy8tTenq6brjhhoovHAAAANWW5SHX4XCof//+SkhI0N69e7V+\n/XqlpKS4X0+ZnZ2t/Px89/LffvutHA6HQkJCSmwrOjra/S7zb7/9VuPHj1dwcLCio6Or7HgAAABg\nPctDriTFx8crPDxcw4YN07Rp0zR27Fj17NlTkhQVFeXxWsbs7GzVqlWr1O2MHz9ed9xxh8aNG6ff\n//73KioqUlJS0mW9khIAAAD2US1e61tdFI/xYEwuAABA9XS5ea1a9OQCAAAAFYmQCwAAANsh5AIA\nAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2\nCLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2CLkA\nAACwHUIuAAAAbIeQCwDABbZt26Zt27ZZXQaAX8nb6gIAAKguCgoKlJSUJC8vL0VGRsrX19fqkgCU\nEz25AAD8n1WrVikrK0uZmZlavXq11eUA+BUIuQAASCWC7apVq5SZmWlhRQB+DUIuAACSkpKSVFBQ\n4J4uHroA4MpEyAUAAIDtEHIBAJAUFxfncaOZr6+v4uLiLKwIwK9ByAUAQFKjRo107733uqcHDhyo\nRo0aWVgRgF+DkAsAwP8ZOHCggoKCSgReAFcenpMLAMD/uXCIAs/IBa5shFwAAC7QoUMHq0sAUAEY\nrgAAAADbIeQCAADAdgi5AAAAsB1CLgAAAGyHkAsAAADbIeQCAADAdgi5AAAAsB1CLgAAAGynWoTc\ngoICTZo0Se3bt1eXLl2UkpJS6nJDhw6V0+ks8d9f//pX9zLvvvuuevXqpcjISI0ePVqnTp2qqsMA\nAABANVEtQu6sWbO0f/9+LVu2TAkJCVqwYIHef//9EsstXLhQmzdvdv+3cOFC+fr6asiQIZKkPXv2\naPLkyRozZoxef/11nT59WvHx8VV9OAAAALCY5a/1dblcWrVqlZKTk909s7GxsVq+fLl69+7tsWzt\n2rXdPxcVFen555/Xww8/rFtuuUWStGLFCt11113q16+fJOnZZ59Vt27ddPToUTVu3LjqDgoAAOAX\nbNu2TRKvkq4slvfkpqamqrCwUJGRke62tm3bas+ePT+73urVq3X69GnFxsa623bt2qX27du7pxs1\naqTg4GDt3r274gsHAAAop4KCAiUlJWnJkiUqKCiwuhxbsjzkHj9+XIGBgfL2/m+ncr169ZSfn/+z\n42mXLl2q4cOHy9/f32NbDRs29Fiufv36yszMrPjCAQAAymnVqlXKyspSZmamVq9ebXU5tmR5yHW5\nXPL19fVoK56+1F82W7Zs0ffff6/77rvPo/3HH38sdVv8hQQAAKqLi4PtqlWr6JCrBJaHXD8/vxIh\ntHj6wl7aC73//vvq0qWLxxjdn9uWw+GowIoBAADKLykpySOvFA9dQMWyPOQGBQUpJydHRUVF7rbs\n7Gw5HI4SIbbYJ598op49e5Zob9iwobKzsz3asrOzSwxhAAAAgL1ZHnLDwsLk7e2tXbt2udt27Nih\n8PDwUpc/deqUjhw5ojZt2pSYFxkZqS+++MI9fezYMWVmZioiIqLiC4ek83eGFt8dClQmrjUAdhEX\nF+cxvNLX11dxcXEWVmRPlodch8Oh/v37KyEhQXv37tX69euVkpKiYcOGSTrfE5ufn+9e/ttvv5XD\n4VBISEiJbf3hD3/Q2rVrtWrVKqWmpmrChAnq1q0bjw+rJNwZiqrCtQbATho1aqR7773XPT1w4EA1\natTIworsyfKQK0nx8fEKDw/XsGHDNG3aNI0dO9Y9HCEqKkrr1q1zL5udna1atWqVup3IyEhNnTpV\nCxcu1ODBgxUYGKjp06dXyTFcjbgzFFWFaw2A3QwcOFBBQUElAi8qjpcxxlhdRHXRo0cPSdKHH35o\ncSXVX2ZmpkaNGuXuVfP19dXChQv5SxQVjmsNgF3xMojyudy8Vi16cnHl4c5QVBWuNQB21aFDBwJu\nJSLkAgAAwHYIuSgX7gxFVeFaAwCUByEX5cKdoagqXGsAgPIg5KLcuDMUVYVrDQBQVt5WF4Ar14Vf\nG1/4dTJQ0bjWAABlRcjFr8JdoagqXGsAgLJguAIAAABsh5ALAAAA2yHkAgAAwHYIuQAAALAdQi4A\nAABsh5ALAAAA2yHkAgAAwHYIuQAAALAdQi4AAABsh5ALAAAA2yHkAgAAwHYIuQAAALAdQi4AAABs\nh5ALAAAA2yHkAgAAwHYIuQAAALAdQi4AAABsh5ALAAAA2yHkAgAAwHYIuQAAALAdQi4AAABsh5AL\nAAAA2yHkAgAAwHYIuQAAALAdQi4AAABsh5ALAAAA2yHkAgAAwHYIuQAAALAdQi4AAABsh5ALAAAA\n2yHkAgAAwHYIuQAAALCdahFyCwoKNGnSJLVv315dunRRSkrKJZf9+uuvNXjwYEVERKhfv37aunWr\nx/x27dpKPtUrAAAgAElEQVQpLCxMTqdTTqdTYWFhcrlclX0IAAAAqEa8rS5AkmbNmqX9+/dr2bJl\nysjI0IQJE9S4cWP17t3bY7mzZ89qxIgR6tGjh2bNmqU1a9Zo9OjReu+991S3bl1lZWUpNzdX69ev\nl8PhcK/n7+9f1YcEAAAAC1nek+tyubRq1SpNnjxZTqdTPXv2VGxsrJYvX15i2TfffFMBAQF6+umn\n1aRJE40ZM0bNmzfXvn37JElpaWlq0KCBGjdurHr16rn/AwAAwNXF8p7c1NRUFRYWKjIy0t3Wtm1b\nLV68uMSy27dvV/fu3T3a3njjDffPBw4cUPPmzSutVgAAAFwZLO/JPX78uAIDA+Xt/d+8Xa9ePeXn\n5+vUqVMeyx45ckR16tTRU089paioKA0aNEg7d+50zz948KBcLpeGDh2qqKgoxcXF6dChQ1V1KAAA\nAKgmLA+5LpdLvr6+Hm3F0wUFBR7teXl5Wrp0qRo2bKilS5eqXbt2GjFihLKysiSdH65w5swZjRo1\nSosWLZLD4dDw4cOVl5dXNQcDAACAasHy4Qp+fn4lwmzx9MU3jNWsWVNhYWEaPXq0JMnpdGrz5s1a\nu3at4uLilJycrHPnzrnXmzNnjqKjo/XRRx/pnnvuqYKjAQAAQHVgecgNCgpSTk6OioqKVKPG+Y7l\n7OxsORwO1a5d22PZBg0a6IYbbvBoa968uY4dOyZJ8vHxkY+Pj3uer6+vQkJC3D29AAAAuDpYPlwh\nLCxM3t7e2rVrl7ttx44dCg8PL7FsZGSkUlNTPdrS0tIUEhIiSerVq5fWrFnjnpeXl6f09PQSwRgA\nAAD2ZnnIdTgc6t+/vxISErR3716tX79eKSkpGjZsmKTzvbr5+fmSpEGDBunrr7/WggULdPjwYb3w\nwgvKyMhQ3759JUnR0dGaP3++tm3bpm+//Vbjx49XcHCwoqOjLTs+AAAAVD3LQ64kxcfHKzw8XMOG\nDdO0adM0duxY9ezZU5IUFRWldevWSZKuv/56JScna8OGDerbt682bdqkpKQkNWzYUJI0fvx43XHH\nHRo3bpx+//vfq6ioSElJSfLy8rLs2AAAAFD1vIwxxuoiqosePXpIkj788EOLKwEAAEBpLjevVYue\nXAAAAKAiEXIBAABgO4RcAAAA2A4hFwAAALZDyAUAAIDtEHIBAABgO4RcAAAA2I631QUAAHCx3Nxc\nZWRkWLJvl8slSfL397dk/yEhIQoICLBk34CdEHIBANVKbm6uRowYodzcXKtLsURAQICSk5MJusCv\nxHAFAAAA2A49uQCAaqW4J9OK4Qrp6elKTEyUJI0ZM0bNmjWr8hoYrgBUDEIuAKDaCQgIUGhoqKU1\nNGvWzPIaAJQfwxUAAABgO4RcAAAA2A4hFwAAALZDyAUAAIDtEHIBAABgO4RcAAAA2A4hFwAAALZD\nyAUAAIDtEHIBAABgO4RcAAAA2A4hFwAAALZDyAUAAIDtEHIBAABgO95WF4BfLzc3VxkZGZbs2+Vy\nSZL8/f0t2X9ISIgCAgIs2ffViGuNaw0ArhSE3Ctcbm6uRowYodzcXKtLsURAQICSk5MJH1WAa41r\nDQCuJAxXAAAAgO3Qk3uFK+5dsuIr5PT0dCUmJkqSxowZo2bNmlV5DXyFXHW41rjWAOBKQsi1gYCA\nAIWGhlpaQ7NmzSyvAZWPaw0AcKVguAIAAABsh5ALAAAA2yHkAgAAwHYIuQAAALAdQi4AAABsh5AL\nAAAA2yHkAgAAwHYIuQAAALAdQi4AAABsh5ALAAAA26kWIbegoECTJk1S+/bt1aVLF6WkpFxy2a+/\n/lqDBw9WRESE+vXrp61bt3rMf/fdd9WrVy9FRkZq9OjROnXqVGWXDwAAgGqmWoTcWbNmaf/+/Vq2\nbJkSEhK0YMECvf/++yWWO3v2rEaMGKGbbrrJHWZHjx6tkydPSpL27NmjyZMna8yYMXr99dd1+vRp\nxcfHV/XhAAAAwGKWh1yXy6VVq1Zp8uTJcjqd6tmzp2JjY7V8+fISy7755psKCAjQ008/rSZNmmjM\nmDFq3ry59u3bJ0lasWKF7rrrLvXr108333yznn32WW3atElHjx6t6sMCAACAhbytLiA1NVWFhYWK\njIx0t7Vt21aLFy8usez27dvVvXt3j7Y33njD/fOuXbv0yCOPuKcbNWqk4OBg7d69W40bN66E6gEA\nwJUsNzdXGRkZluzb5XJJkvz9/S3Zf0hIiAICAizZd1WwPOQeP35cgYGB8vb+byn16tVTfn6+Tp06\npTp16rjbjxw5olatWumpp57Shg0bFBISovHjx6tNmzbubTVs2NBj+/Xr11dmZmbVHAwAALhi5Obm\nasSIEcrNzbW6FEsEBAQoOTnZtkG3WgxX8PX19Wgrni4oKPBoz8vL09KlS9WwYUMtXbpU7dq104gR\nI5SVlSVJ+vHHH0vd1sXbAQAAgL1Z3pPr5+dXIoQWT1/cfV+zZk2FhYVp9OjRkiSn06nNmzdr7dq1\niouLu+S2HA5HJR4BAAC4EhX3ZFoxXCE9PV2JiYmSpDFjxqhZs2ZVXgPDFSpZUFCQcnJyVFRUpBo1\nzncsZ2dny+FwqHbt2h7LNmjQQDfccINHW/PmzXXs2DFJUsOGDZWdne0xPzs7u8QQBgAAAOl80A0N\nDbW0hmbNmllegx1ZPlwhLCxM3t7e2rVrl7ttx44dCg8PL7FsZGSkUlNTPdrS0tIUEhLinv/FF1+4\n5x07dkyZmZmKiIiopOoBAABQHVkech0Oh/r376+EhATt3btX69evV0pKioYNGybpfE9sfn6+JGnQ\noEH6+uuvtWDBAh0+fFgvvPCCMjIy1LdvX0nSH/7wB61du1arVq1SamqqJkyYoG7duvFkBQAAgKuM\n5SFXkuLj4xUeHq5hw4Zp2rRpGjt2rHr27ClJioqK0rp16yRJ119/vZKTk7Vhwwb17dtXmzZtUlJS\nkns4QmRkpKZOnaqFCxdq8ODBCgwM1PTp0y07LgAAAFjD8jG50vne3BkzZmjGjBkl5l08PKF169Z6\n8803L7mtmJgYxcTEVHiNAAAAuHJUi55cAAAAoCJVi55cAED14nK5dPjwYavLqHLp6eml/nw1adq0\nqWVv4AIqEiEXAFDC4cOHNW7cOKvLsFTxM0yvNnPmzOFxVrAFhisAAADAdujJBQD8rNuvuVZ1a9a0\nuowq85MxkiQfLy+LK6k6JwsLtTHvrNVlABWKkAsA+Fl1a9ZUkLeP1WUAQJkwXAEAAAC2Q8gFAACA\n7RByAQAAYDuEXAAAANgOIRcAAAC2Q8gFAACA7RByAQAAYDuEXAAAANgOIRcAAAC2Q8gFAACA7RBy\nAQAAYDuEXAAAANgOIRcAAAC2Q8gFAACA7RByAQAAYDuEXAAAANgOIRcAAAC2Q8gFAACA7RByAQAA\nYDuEXAAAANgOIRcAAAC2Q8gFAACA7RByAQAAYDuEXAAAANgOIRcAAAC2Q8gFAACA7RByAQAAYDuE\nXAAAANiOt9UFAACqt5OF56wuAZWMcww7IuQCAEpwuVzunzfm5VpYCaraheceuJIxXAEAAAC2Q08u\nAKAEf39/98+3XxOgujX558LOThaec/fYX3jugSsZ/69VAVwulw4fPmx1GVUuPT291J+vJk2bNq3S\nfxC41rjWrFC3preCvH0s2TcAlBchtwIcPnxY48aNs7oMSyUmJlpdgiXmzJmj0NDQKtsf1xrXGgDg\n8lSLkFtQUKApU6bogw8+kMPh0EMPPaQHH3yw1GVHjhypjz76SF5eXjLGyMvLSy+99JKio6MlSe3a\ntVNubq6MMZIkLy8v7dy5k69fAAAAriLVIuTOmjVL+/fv17Jly5SRkaEJEyaocePG6t27d4ll09LS\nNHfuXN12223uttq1a0uSsrKylJubq/Xr18vhcLjnV2XA9Qtur5p+gVW2P6uZop8kSV41rp6vMgvz\nc5R/bLvVZSigTX3VrO1rdRlVxpwrkiR5eV8998sWnilQ7s5sq8sAgCuS5SHX5XJp1apVSk5OltPp\nlNPpVGxsrJYvX14i5BYUFCgjI0Ph4eGqV69eiW2lpaWpQYMGaty4cVWVX0JNv0DV9C9ZG1DRatb2\nlU9dxy8vCADAVcjyLpHU1FQVFhYqMjLS3da2bVvt2bOnxLLfffedvLy81KRJk1K3deDAATVv3ryy\nSgUAAMAVwvKQe/z4cQUGBsrb+7+dyvXq1VN+fr5OnTrlsezBgwd17bXX6sknn1RUVJTuu+8+ffzx\nxx7zXS6Xhg4dqqioKMXFxenQoUNVdSgAAACoJiwPuS6XS76+nuMKi6cLCgo82tPS0pSfn68uXboo\nOTlZ0dHRGjlypP7f//t/7vlnzpzRqFGjtGjRIjkcDg0fPlx5eXlVczAAAACoFiwfk+vn51cizBZP\nX3zD2OjRozVs2DDVqlVLkhQaGqp9+/Zp5cqVmjp1qpKTk3Xu3Dn3enPmzFF0dLQ++ugj3XPPPVVw\nNAAAAKgOLA+5QUFBysnJUVFRkWrUON+xnJ2dLYfD4X5qwoWKA26xFi1a6ODBg5IkHx8f+fj89y5/\nX19fhYSEKCsrqxKPAAAAlBcvueElN5XF8pAbFhYmb29v7dq1S23atJEk7dixQ+Hh4SWWjY+Pl5eX\nl6ZPn+5uS01NdT8gvVevXho1apRiYmIkSXl5eUpPT9cNN9xQBUcCAADKipfc8JKbymL5mFyHw6H+\n/fsrISFBe/fu1fr165WSkqJhw4ZJOt+rm5+fL0nq3r273nnnHa1Zs0aHDx/WggULtHPnTg0dOlSS\nFB0drfnz52vbtm369ttvNX78eAUHB7tfFAEAAICrg+U9udL5Htqnn37aPd527Nix6tmzpyQpKipK\nM2fOVExMjHr16qWEhAQtWrRImZmZuvHGG7V06VIFBwdLksaPHy8fHx+NGzdOP/zwgzp16qSkpCR5\neXlZeXgAAOAy3BDSSdc4rp4XKhUWnn+hUs2aV88LlfJ+zFFaxudVsq9qEXIdDodmzJihGTNmlJiX\nmprqMT1w4EANHDiw1O34+vpqwoQJmjBhQqXUCQAAKs81jkDVuqaB1WXAJiwfrgAAAABUNEIuAAAA\nbIeQCwAAANsh5AIAAMB2yhVyU1NTFR8fr0GDBikrK0srVqzQ1q1bK7o2AAAAoFzKHHL37dun3//+\n98rIyNC+fftUUFCgr776SiNGjNCmTZsqo0YAAACgTMoccufMmaMHH3xQy5Ytc79C9+9//7uGDBly\n1b6xAwAAANVLuXpyi1+be6EhQ4bo4MGDFVIUAAAA8GuUOeT6+Pjo7NmzJdqPHTsmf3//CikKAAAA\n+DXKHHJ79uypefPm6cyZM+62gwcP6plnntHtt99ekbUBAAAA5VLmkDthwgTl5ubqtttuk8vl0u9+\n9zv16dNHNWvW1Pjx4yujRgAAAKBMvMu6gpeXl1577TV9/vnn2r9/v4qKinTzzTerS5cuqlGDx+4C\nAADAemUOuTExMZo3b546deqkTp06VUZNAAAAwK9S5pDrcrnkcDgqoxYAQDV0srDQ6hKq1E/GSJJ8\nvLwsrqTqXG3nGFeHMofcBx54QGPGjNGQIUPUtGnTEoG3ffv2FVYcAMB6G/NKPlEHAKq7Mofc5557\nTpI0bdq0EvO8vLz01Vdf/fqqAAAAgF+hzCH3ww8/rIw6AADVSNOmTTVnzhyry6hy6enp7rd3jhkz\nRs2aNbO4oqrXtGlTq0sAKkSZQ27jxo0lSWfPnlVaWpp8fHzUpEkTXXvttRVeHADAGv7+/goNDbW6\nDEs1a9bsqv8MgCtZmUNuUVGRZs2apVdffVXnzp2TMUa+vr66//77NWnSJHldRQP1S1OYf9rqElDJ\nqss5PnemwOoSUMk4xwBQfmUOuYsXL9bq1av15JNPqkOHDioqKtL27du1cOFCBQUFKTY2tjLqrNZc\nLpf75/xj2yysBFXtwnNf1fvL25ldpfuGtar6WgOskPdjjtUloJJV5Tkuc8h94403lJCQoL59+7rb\nbrnlFtWtW1eJiYlXZcgFAADlc+EfcGkZn1tYCapaZf/xXuaQe+LECUVERJRoj4iI0LFjxyqkqCuN\nv7+/+2e/4A6q6XedhdWgshXmn3b32F947qvChfu7pk19edf2rdL9o2qdO1Pg7rGv6msNAK50ZQ65\nzZs312effVbi7svNmze7b0q7mtX0u041/etZXQauAt61feVTlxezALiyXfgH3A0hnXSNI9DCalDZ\n8n7McffYV/Yf72UOuQ8++KCeeuopHTlyRG3atJEkffHFF1qxYoXGjx9f4QUCAICrwzWOQNW6poHV\nZcAmyhxyY2JilJOTo6VLlyo5OVmSVL9+fT3++OMaMmRIhRcIAAAAlFWZQ64kDR8+XIMHD9bZs2dl\njNG5c+cUFBRU0bUBAAAA5VKjrCucPHlSDzzwgBYuXKi6deuqXr16GjBggB566CGdPl09nh8KAACA\nq1uZQ+4zzzwjl8ule+65x922ZMkS/fDDD5o1a1aFFgcAAACUR5lD7qeffqpp06bp5ptvdre1bNlS\nCQkJ2rhxY0XWBgAAAJRLmUNuYWGhjDEl2n18fHgjDwAAAKqFMofc9u3b67nnntPZs2fdbWfPntUL\nL7yg9u3bV2hxAAAAQHmU+ekK8fHxGjx4sLp27armzZtLkg4dOqTrrrvO/UgxAAAAwEplDrlNmzbV\nunXr9O9//1vffPONvL29dffdd2vo0KHy8/OrjBoBAACAMinzcIUzZ85o9uzZioyM1KRJk/T1119r\n7ty5iomJ0ZEjRyqjRgAAAKBMyhxyZ8yYoS1btsjb21sffPCBdu7cqdmzZ6t58+aaPXt2ZdQIAAAA\nlEmZhyts2rRJCxcuVIsWLbRkyRL99re/Vd++fRUaGsprfQEAAFAtlLknNy8vT8HBwZKkzZs367e/\n/a0kyeFwqLCwsGKrAwAAAMqhzD25LVq00MaNGxUcHKzjx4+ra9eukqTXX39dLVq0qPACAQAAgLIq\nc8j905/+pDFjxuinn35Snz591Lx5c82YMUMrVqzQwoULK6NGAAAAoEzKPFwhOjpamzZt0ltvvaU5\nc+ZIku655x6tXbtW0dHR5SqioKBAkyZNUvv27dWlSxelpKRcctmRI0fK6XQqLCzM/b+bNm1yz3/3\n3XfVq1cvRUZGavTo0Tp16lS5agIAAMCVq8w9uZJUp04d1alTxz196623/qoiZs2apf3792vZsmXK\nyMjQhAkT1LhxY/Xu3bvEsmlpaZo7d65uu+02d1vt2rUlSXv27NHkyZM1depUOZ1OTZs2TfHx8Xrp\npZd+VX1lUZifU2X7qg5M0U+SJK8aPhZXUnWqyzkuPFNgdQlVypwrkiR5eZf5b/Mr1tV2jgGgIpUr\n5FYkl8ulVatWKTk5WU6nU06nU7GxsVq+fHmJkFtQUKCMjAyFh4erXr16Jba1YsUK3XXXXerXr58k\n6dlnn1W3bt109OhRNW7cuEqOJ//Y9irZD5C7M9vqEgAAqLYs7xJJTU1VYWGhIiMj3W1t27bVnj17\nSiz73XffycvLS02aNCl1W7t27VL79u3d040aNVJwcLB2795d8YUDAACg2rK8J/f48eMKDAyUt/d/\nS6lXr57y8/N16tQpj2ERBw8e1LXXXqsnn3xSW7duVXBwsMaMGeN+wsPx48fVsGFDj+3Xr19fmZmZ\nlXoMTZs2dY9Pvpqkp6crMTFRkjRmzBg1a9bM4oqqXtOmTat8f1xrXGsAgF9mech1uVzy9fX1aCue\nLijwHI+Wlpam/Px8denSRXFxcfrggw80cuRIvf7662rZsqV+/PHHUrd18XYqmr+/v0JDQyt1H9Vd\ns2bNrvrPoCpwrXGtAXaW92P1uOehqhQWnr+vpWbNq+e+lqo8x5aHXD8/vxIhtHja39/fo3306NEa\nNmyYatWqJUkKDQ3Vvn37tHLlSk2dOvWS23I4HJV4BAAAoCKkZXxudQmwEcvH5AYFBSknJ0dFRUXu\ntuzsbDkcDvdTEy5UHHCLtWjRQt9//70kqWHDhsrO9rwZJzs7u8QQBgAAANib5T25YWFh8vb21q5d\nu9SmTRtJ0o4dOxQeHl5i2fj4eHl5eWn69OnuttTUVPdXl5GRkfriiy8UExMjSTp27JgyMzMVERFR\nBUcCAADKinsNuNegslgech0Oh/r376+EhARNnz5dWVlZSklJ0cyZMyWd74mtVauW/Pz81L17d/35\nz39Whw4d1KZNG7399tvauXOnpk2bJkn6wx/+oAceeEAREREKDw/X9OnT1a1btyp7fBgAACgb7jXg\nXoPKYnnIlc730D799NPu8bZjx45Vz549JUlRUVGaOXOmYmJi1KtXLyUkJGjRokXKzMzUjTfeqKVL\nl+r666+XdL4nd+rUqXrhhRd0+vRpRUVFuQMwAAAArh7VIuQ6HA7NmDFDM2bMKDEvNTXVY3rgwIEa\nOHDgJbcVExPjHq4AAACAq5PlN54BAAAAFY2QCwAAANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAA\nANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh\n5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIA\nAMB2CLkAAACwHUIuAAAAbIeQCwAAANvxtroAAAAulpubq4yMjCrfb3p6eqk/V6WQkBAFBARYsm/A\nTgi5AIBqJTc3VyNGjFBubq6ldSQmJlqy34CAACUnJxN0gV+J4QoAAACwHXpyAQDVSnFPphXDFSTJ\n5XJJkvz9/S3ZP8MVgIpByAUAVDsBAQEKDQ21ugwAVzCGKwAAAMB2CLkAAACwHUIuAAAAbIeQCwAA\nANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2qkXILSgo0KRJk9S+fXt16dJFKSkp\nv7hORkaGWrdure3bt3u0t2vXTmFhYXI6nXI6nQoLC3O/ohEAAABXh2rxWt9Zs2Zp//79WrZsmTIy\nMjRhwgQ1btxYvXv3vuQ6U6ZM0Y8//ujRlpWVpdzcXK1fv14Oh8PdbtX7xwEAAGANy0Ouy+XSqlWr\nlJyc7O59jY2N1fLlyy8Zct9++23l5eWVaE9LS1ODBg3UuHHjyi4bAAAA1ZjlwxVSU1NVWFioyMhI\nd1vbtm21Z8+eUpc/deqU5s6dq2nTpskY4zHvwIEDat68eWWWCwAAgCuA5SH3+PHjCgwMlLf3fzuV\n69Wrp/z8fJ06darE8jNnztSAAQPUokWLEvMOHjwol8uloUOHKioqSnFxcTp06FBllg8AAIBqyPKQ\n63K55Ovr69FWPF1QUODR/tlnn+nLL7/UY489Vuq20tLSdObMGY0aNUqLFi2Sw+HQ8OHDSx3aAAAA\nAPuyfEyun59fiTBbPH3hDWP5+flKSEjQlClTSoTiYsnJyTp37px7vTlz5ig6OlofffSR7rnnnko6\nAuvl5uYqIyOjyvebnp5e6s9VKSQkRAEBAZbs+2rEtca1BgBXCstDblBQkHJyclRUVKQaNc53LGdn\nZ8vhcKh27dru5fbs2aOMjAyNGTPGYyzuww8/rJiYGE2ZMkU+Pj7y8fFxz/P19VVISIiysrKq7oCq\nWG5urkaMGKHc3FxL60hMTLRkvwEBAUpOTiZ8VAGuNa41ALiSWB5yw8LC5O3trV27dqlNmzaSpB07\ndig8PNxjuYiICL3//vsebb169dIzzzyjTp06uadHjRqlmJgYSVJeXp7S09N1ww03VMGRAAAAoLqw\nPOQ6HA71799fCQkJmj59urKyspSSkqKZM2dKOt+rW6tWLfn5+alJkyYl1m/YsKHq1q0rSYqOjtb8\n+fN1/fXXq06dOnrhhRcUHBys6OjoKj2mqlTcu2TFV8iS3C/asOpZxHyFXHW41rjWAOBKYnnIlaT4\n+Hg9/fTTGjZsmGrVqqWxY8eqZ8+ekqSoqCjNnDnT3Tt7IS8vL4/p8ePHy8fHR+PGjdMPP/ygTp06\nKSkpqcRydhMQEKDQ0FCry8BVgGsNAHClqBYh1+FwaMaMGZoxY0aJeampqZdc76uvvvKY9vX11YQJ\nEzRhwoQKrxEAAABXDssfIQYAAABUNEIuAAAAbIeQCwAAANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQ\nCwAAANsh5AIAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2CLkAAACwHW+rCwAAALBKbm6u\nMjIyqny/6enppf5clUJCQhQQEGDJvqsCIRcAAFyVcnNzNWLECOXm5lpaR2JioiX7DQgIUHJysm2D\nLsMVAAAAYDv05AIAgKtScU+mFcMVJMnlckmS/P39Ldk/wxUAAABsKiAgQKGhoVaXgUrAcAX8Ktu2\nbdO2bdusLgNXAa41AEBZ0JOLcisoKFBSUpK8vLwUGRkpX19fq0uCTXGtAQDKip5clNuqVauUlZWl\nzMxMrV692upyYGNcawCAsiLkolwuDhurVq1SZmamhRXBrrjWAADlQchFuSQlJamgoMA9Xfx1MlDR\nuNYAAOVByAUAAIDtEHJRLnFxcR43//j6+iouLs7CimBXXGsAgPIg5KJcGjVqpHvvvdc9PXDgQDVq\n1MjCimBXXGsAgPIg5KLcBg4cqKCgoBIhBKhoXGsAgLLiObkotwu/Nua5pahMXGsAgLIi5OJX6dCh\ng9Ul4CrBtQYAKAuGKwAAAMB2CLkAAACwHUIuAAAAbIeQCwAAANsh5AIAAMB2CLkAAACwHUIuAAAA\nbIeQCwAAANsh5AIAAMB2CLkAAACwHUIuAAAAbKdahNz/396dx+WU9n8A/6QmWRpZw/CMbXRH6m5F\nRc9kGWshL4QsoxrzZJkxY4lMHkYyWUbK0ixlG1uyDMYYDMYyLUgpaddCkT3Sev3+8Os8biXL4M7x\neRtK4+AAACAASURBVL9eXi/nnOtc1/c659x3365znVNRURFmz54NS0tLdOvWDcHBwc/cJysrC6am\npoiMjFRZv3fvXvTq1QtKpRKTJk3CrVu3XlfYRERERFRNVYskd/HixYiPj8eGDRvg7e2NgIAAHDx4\nsMp95s2bh4cPH6qsi4mJgZeXFyZPnoxt27bhzp078PT0fJ2hExEREVE1pPYkt6CgAKGhofDy8oJC\noUDPnj3h6uqKjRs3PnWfPXv24MGDBxXWb9q0CX379oWDgwPat28PPz8/HDt2DNnZ2a+zC0RERERU\nzag9yU1ISEBpaSmUSqW0ztzcHDExMZWWv3XrFpYuXYoFCxZACKGyLTo6GpaWltJy06ZN0axZM5w/\nf/71BE9ERERE1ZLak9zr169DT08PWlpa0rqGDRuisLCw0vm0vr6+GDx4MNq2bVtpXU2aNFFZ16hR\nI+Tk5Lz6wImIiIio2lJ7kltQUABtbW2VdeXLRUVFKutPnTqFc+fO4T//+U+ldT18+LDSup6sh4iI\niIjkTe1Jbs2aNSskoeXLtWrVktYVFhbC29sb3t7eFRLZZ9Wlo6PziqMmIiIioupM69lFXi99fX3c\nvn0bZWVlqFHjUc6dl5cHHR0dvP/++1K5mJgYZGVlYfLkySpzcd3c3DBo0CDMmzcPTZo0QV5enkr9\neXl5FaYwEBEREZG8qT3JNTQ0hJaWFqKjo2FmZgYAiIqKgpGRkUo5ExOTCq8V69WrFxYuXIiuXbsC\nAJRKJc6cOYNBgwYBAK5evYqcnByYmJi8gZ4QERERUXWh9iRXR0cHjo6O8Pb2ho+PD3JzcxEcHAxf\nX18Aj0ZidXV1UbNmTbRs2bLC/k2aNEGDBg0AAM7OzhgzZgxMTExgZGQEHx8ffPzxx/jggw/eaJ+I\niIiISL3UPicXADw9PWFkZISxY8diwYIFmDp1Knr27AkAsLW1xW+//VbpfhoaGirLSqUS8+fPR2Bg\nIEaOHAk9PT34+Pi89viJiIiIqHrREE++bPYd1qNHDwDA4cOH1RwJEREREVXmefO1ajGSS0RERET0\nKjHJJSIiIiLZYZJLRERERLLDJJeIiIiIZIdJLhERERHJDpNcIiIiIpIdJrlEREREJDtMcomIiIhI\ndpjkEhEREZHsMMklIiIiItlhkktEREREssMkl4iIiIhkh0kuEREREckOk1wiIiIikh0muUREREQk\nO0xyiYiIiEh2mOQSERERkewwySUiIiIi2WGSS/9IREQEIiIi1B0GvQN4rRER0YvQUncA9PYqKipC\nUFAQNDQ0oFQqoa2tre6QSKZ4rRER0YviSC69tNDQUOTm5iInJwc7duxQdzgkY7zWiIjoRTHJpZfy\nZLIRGhqKnJwcNUZEcsVrjYiIXgaTXHopQUFBKCoqkpbLbycTvWq81oiI6GUwySUiIiIi2WGSSy/F\n3d1d5eEfbW1tuLu7qzEikitea0RE9DKY5NJLadq0KZycnKTloUOHomnTpmqMiOSK1xoREb0MJrn0\n0oYOHQp9ff0KSQjRq8ZrjYiIXhTfk0sv7fHbxnxvKb1OvNaIiOhFMcmlf8TKykrdIdA7gtcaERG9\nCE5XICIiIiLZYZJLRERERLLDJJeIiIiIZIdJLhERERHJDpNcIiIiIpIdJrlEREREJDtMcomIiIhI\ndpjkEhEREZHsMMklIiIiItlhkktEREREslMtktyioiLMnj0blpaW6NatG4KDg59ads+ePfjkk09g\nYmICZ2dnxMTEqGy3sLCAoaEhFAoFFAoFDA0NUVBQ8Lq7QERERETViJa6AwCAxYsXIz4+Hhs2bEBW\nVhZmzpyJDz74AL1791YpFxUVBS8vL/j4+ECpVGLTpk1wc3PD0aNHUatWLeTm5uL+/fs4dOgQdHR0\npP1q1ar1prtERERERGqk9pHcgoIChIaGwsvLCwqFAj179oSrqys2btxYoWxeXh48PDwwYMAAtGjR\nAh4eHrhz5w6Sk5MBAKmpqWjcuDE++OADNGzYUPpHRERERO8WtY/kJiQkoLS0FEqlUlpnbm6OtWvX\nVijbp08f6f+FhYUICQlBo0aN0K5dOwBAcnIyWrVq9dpjJiIiIqLqTe1J7vXr16Gnpwctrf+F0rBh\nQxQWFuLWrVuoX79+hX1Onz6NCRMmAACWLFkiTUdISUlBQUEBXFxckJaWhg4dOmD27NlMfImIiIje\nMdViuoK2trbKuvLloqKiSvcxMDBAWFgYpkyZgpkzZ0oPn6WmpuLu3bvw8PDA6tWroaOjg3HjxuHB\ngwevtxNEREREVK2ofSS3Zs2aFZLZ8uWnPTDWoEEDNGjQAAqFAtHR0di8eTOMjY3x008/oaSkRNpv\nyZIlsLOzw59//on+/fu/3o4QERERUbWh9pFcfX193L59G2VlZdK6vLw86Ojo4P3331cpGxsbi/j4\neJV1bdu2xa1btwAA7733nkpirK2tjRYtWiA3N/c19oCIiIiIqhu1J7mGhobQ0tJCdHS0tC4qKgpG\nRkYVyoaGhmLp0qUq6+Li4qQHz3r16oVdu3ZJ2x48eIDLly+jTZs2ryl6IiIiIqqO1J7k6ujowNHR\nEd7e3oiNjcWhQ4cQHByMsWPHAng0qltYWAgAGD58OMLDw7FhwwZcvnwZ/v7+iI2Nlcra2dnB398f\nERERSEpKwowZM9CsWTPY2dmprX9ERERE9OapPckFAE9PTxgZGWHs2LFYsGABpk6dip49ewIAbG1t\n8dtvvwEAOnTogMDAQGzfvh2Ojo7466+/8PPPP6Nx48YAgBkzZuCTTz7B119/jWHDhqGsrAxBQUHQ\n0NBQW9+IiIiI6M3TEEIIdQdRXfTo0QMAcPjwYTVHQkRERESVed58rVqM5BIRERERvUpMcomIiIhI\ndpjkEhEREZHsMMklIiIiItlhkktEREREssMkl4iIiIhkh0kuEREREckOk1wiIiIikh0muUREREQk\nO0xyiYiIiEh2mOQSERERkewwySUiIiIi2WGSS0RERESywySXiIiIiGSHSS4RERERyQ6TXCIiIiKS\nHSa5RERERCQ7THKJiIiISHaY5BIRERGR7DDJJSIiIiLZYZJLRERERLLDJJeIiIiIZIdJLhERERHJ\nDpNcIiIiIpIdJrlEREREJDtMcomIiIhIdpjkEhEREZHsMMklIiIiItlhkktEREREssMkl4iIiIhk\nh0kuEREREckOk1wiIiIikh0muURERERqEBERgYiICHWHIVta6g6AiIiI6F1TVFSEoKAgaGhoQKlU\nQltbW90hyQ5HcomIiIjesNDQUOTm5iInJwc7duxQdziyxCSXiIiI6A16MrENDQ1FTk6OGiOSJya5\nRERERG9QUFAQioqKpOXyqQv0ajHJJSIiIiLZYZJLRERE9Aa5u7urPGimra0Nd3d3NUYkT0xyiYiI\niN6gpk2bwsnJSVoeOnQomjZtqsaI5KlaJLlFRUWYPXs2LC0t0a1bNwQHBz+17J49e/DJJ5/AxMQE\nzs7OiImJUdm+d+9e9OrVC0qlEpMmTcKtW7ded/hEREREL2To0KHQ19evkPDSq1MtktzFixcjPj4e\nGzZsgLe3NwICAnDw4MEK5aKiouDl5YXJkydj3759UCqVcHNzQ0FBAQAgJiZG2r5t2zbcuXMHnp6e\nb7o7RERERFUqn6Lg5ubGd+S+JmpPcgsKChAaGgovLy8oFAr07NkTrq6u2LhxY4WyeXl58PDwwIAB\nA9CiRQt4eHjgzp07SE5OBgBs2rQJffv2hYODA9q3bw8/Pz8cO3YM2dnZb7pbRERERFWysrKClZWV\nusOQLbUnuQkJCSgtLYVSqZTWmZubV5iGAAB9+vTBZ599BgAoLCxESEgIGjVqhHbt2gEAoqOjYWlp\nKZVv2rQpmjVrhvPnz7/mXhARERFRdaL2P+t7/fp16OnpQUvrf6E0bNgQhYWFuHXrFurXr19hn9On\nT2PChAkAgCVLlqBWrVpSXU2aNFEp26hRI75gmYiIiOgdo/Ykt6CgoMJclPLlx1+U/DgDAwOEhYXh\n6NGjmDlzJlq0aAFjY2M8fPiw0rqeVs+Trl+/jpKSEvTo0eMlekJEREREr9vVq1dVBkefRu1Jbs2a\nNSskoeXL5SO0T2rQoAEaNGgAhUKB6OhobN68GcbGxk+tS0dH57li0dbWhhDiJXpBRERERG+Cpqbm\ncz2sp/YkV19fH7dv30ZZWRlq1Hg0RTgvLw86Ojp4//33VcrGxsZCU1MTHTp0kNa1bdsWKSkpAIAm\nTZogLy9PZZ+8vLwKUxieJioq6p90hYiIiIiqCbU/eGZoaAgtLS1ER0dL66KiomBkZFShbGhoKJYu\nXaqyLi4uTnrwTKlU4syZM9K2q1evIicnByYmJq8peiIiIiKqjjTnzZs3T50BaGlp4erVq9i8eTM6\ndeqE2NhYLFmyBF9//TXatGmDvLw8aGpqQktLC/r6+vj++++hq6sLPT09rF+/HgcPHoSfnx/q1KmD\nxo0bw9fXF40bN0aNGjXg7e0NAwMDjBgxQp1dJCIiIqI3TENUg0moDx8+xH//+1/8/vvv0NXVhaur\nK1xcXAAACoUCvr6+GDRoEADg2LFjWLp0KTIyMvDRRx/By8tLZaR2165dWLFiBe7cuQNbW1ssWLAA\n9erVU0u/iIiIiEg9qkWSS0RERET0Kql9Ti4RERER0avGJJeIiIiIZIdJLhERERHJDpNcIiIiIpId\nJrlEREREJDtMconotcjOzoZCocCVK1cAPHodYGRk5Btrf+fOnejRo8dzlQ0ICJBeW1iZ+/fvY9eu\nXS8dy7Pqp4oiIiJgaGj4yspVF09+LqoSEREBhUJRZZlffvnlpWN5nvrl6PDhw7Czs4OpqSlOnjwJ\nACgqKsLAgQPf6HdUOXt7++f+fnnW9+jff/+N1NTUl47lTX9Pv25MconotdHQ0FBb2/3790doaOhz\nl68q1uDgYISFhf2jeNR5LN5GZmZmOHHixCsrV100b94cJ0+eRLNmzZ6rfFXXTWRkJObPn/+P4nkX\nr8uVK1eie/fu+O2332BpaYmioiJMmzYNycnJaolnx44d6Nev3yupa9y4cbhx48YrqUsOtNQdABHJ\nlzpfw62trQ1tbW21tU//jJaWFho2bPjKylUXGhoaryzesrKydzJJ/afu3bsHMzMzNG3aFCkpKfjq\nq6/UGk/9+vXV2r6ccSRXxs6cOYORI0dCqVTC1NQU7u7uyMvLAwAcP34cQ4YMgVKpxKBBg3D69Glp\nv927d6Nv375QKpVwdnbGxYsXAQCenp7w9PRUaePxWxv29vZYsmQJbG1tMWTIEACPbgsNHjwYxsbG\nsLS0xFdffYWCgoIq28rJyYGhoaHULgDcvHkTHTt2RGZm5us5WO+Q8tulq1atgpWVFb799lscOnQI\n/fv3h1KpxLBhw1RuV5WWlmLZsmWwtbWFhYUFpk6ditu3bwMAcnNzMWXKFFhZWaFTp04YMmQIzp49\n+4/ic3R0xKZNm6Tl8ePHq9zq37ZtG0aOHAkAyMnJwcSJE6FUKtGjRw8EBARIifXOnTthb28v7Xfh\nwgUMHz4cJiYmcHZ2hr+/v0q9xcXFmD9/PszNzWFjY4OQkBCpnoCAAJXb4kVFRfj222/RpUsXdOnS\nBdOnT8edO3ekulJSUqTP3rhx43Dr1q1/dEyqk/LrZ+/evejevTssLS3h4+OD0tJSBAQEwMPDA6NH\nj0bnzp0RFRX1zGOVkZEBV1dXmJqawt7eHhs2bAAAhIeHq9xKX79+Pezt7WFsbAwnJyecOXMGQMVb\n7rm5uZg6dSo6d+6MLl264Ntvv0VxcTGAR+fSxcUFK1euRJcuXWBpaQlfX9/n6vfChQsxZcoUaXn1\n6tXo1KkTioqKAABpaWkwNjbGw4cPq+zzk9MVbt++jUmTJsHU1BS9evXCli1bKkwh2LJlC7p37w5T\nU1N4enqiuLgY2dnZGDt2LIQQMDQ0lD6zW7ZsQY8ePWBqaooxY8YgMTFRqic/Px/Tpk2DmZkZ+vTp\ng9jY2Ofqu5zY29vjypUr8PT0RI8ePRAZGYkuXbpg69atL/RL+aFDh2BtbS0tnzlzBgqFAhEREdK6\n7t27Sz9bqzovj09XEEJgyZIl0rWzevVq9O7dW+U7OTIyEgMHDoSxsTFcXFxw9epVqR4AGDNmDAIC\nAgAAUVFRcHJygomJCRwcHHDw4EGVfgQEBMDa2hpdu3Z9oTtfbwsmuTKVn5+PiRMnolu3bti/fz9+\n/vlnZGRkYO3atUhOTsZ//vMffPLJJ9izZw/69esHDw8P3LhxA3/99RfmzJmD8ePH49dff0XHjh0x\nceJElJSUPFe7e/fuRUhICBYtWoTMzExMnToVo0aNwoEDB7BixQqcPn0aW7duBYCnttWoUSOYm5vj\nwIEDUr2///47OnTogJYtW76W4/UuOnfuHMLCwjB06FDMnDkTHh4e+PXXX+Hg4AB3d3fpF4rvv/8e\nu3fvxuLFi7F161bcuHED3t7eAIDp06dDCIFt27Zh165daNq0Kf773//+o7hsbW2lHxQlJSU4f/48\nLly4gNLSUgDAyZMnYWdnBwCYNGkSmjRpgt27d8PX1xf79u3DmjVrpLrKR7ny8/Ph5uaGTp06Yffu\n3RgwYACCgoJURsHOnTuHmjVrYteuXXBzc4Ovry9SU1PRr18/jB8/XmX+3rJlyxAXF4cff/wRGzZs\nQH5+PqZOnQrgUQLs7u6ODz/8EDt37kTv3r2la15OAgMDsWLFCgQGBuL333+Hv78/gEe/2Do4OGDd\nunXo1KnTM4/Vp59+irp16yI0NBRz587F8uXLcezYMWhoaEjnJz4+Hn5+fpg3bx4OHDgACwsLfPHF\nF1Is5eWKi4sxZswYFBYWYtOmTVixYgWOHTsGPz8/qey5c+eQnp6OLVu2YO7cuVi/fr3KL/lPY2tr\nKyXWwKPkobS0FBcuXAAAnDp1ChYWFtDR0amyz4/HCwBffvklbt++ja1bt2Lu3LkICAhQ2S6EwMGD\nBxEcHIxVq1bhwIEDCAsLQ/PmzbFy5UpoaGjg5MmTMDU1xZEjRxAYGIhvvvkGu3fvhoWFBcaOHYt7\n9+4BALy9vZGeno5ffvkFc+fORXBw8HOebfnYsWMH9PX14eXlhdDQUIwYMQKzZs1CzZo1X6ieLl26\n4O7du9IUh6ioKNSoUUP6JT8pKQn5+fmwsLB45nl53Jo1a7Bnzx4sX74cISEhOHr0KLKyslTKhIaG\nwtvbGzt27MCdO3ewZMkSaT3waDrGhAkTkJeXh4kTJ8LJyQl79+6Fm5sbPD09pet469at2LBhAxYt\nWoTg4GCEhobK786AIFm6fv26CA4OVlm3dOlSMW7cOLFo0SLh4uKism3FihUiNTVVTJo0ScyaNUta\nX1RUJBYvXizy8vLErFmzVLYJIYSBgYGIiIgQQgjx8ccfi6VLl0rb0tPTxdatW1XKT5s2TcyZM0cI\nIaps65dffhG9e/eWtrm4uFToD72crKwsYWBgIE6cOCGEEGL69OnC19dXpczkyZOldZ07dxY7d+6U\ntiUnJ4uVK1cKIYRYv369yMnJkbYdP35cdOjQQaWd7OxsIYTqtVKVU6dOCWtrayGEENHR0cLR0VF0\n795dnD9/XpSVlYnOnTuLCxcuqJQrd+TIEWFlZSWEECIsLEzY29sLIYTYsmWL6NGjhygrK5PKTps2\nTfocrFy5UtjZ2anUZWlpKfbv3y9tLy9bUFAgjIyMRGJiolT2zp07wtDQUCQmJoo///xTmJubi4cP\nH0rbp06dWuEz97YqP69HjhyR1u3YsUN07dpV+Pv7CxsbG2n9s47VoUOHhJmZmXjw4IG0PSwsTBw/\nflyEh4cLhUIhhBDijz/+EMbGxiIpKUkIIcSDBw/E6dOnRWlpqUq5Q4cOCaVSKe7duyfVd/z4cdGx\nY0fx4MEDERYWJjp06CDu378vbR88eLBYu3btM/td3pekpCRRUlIizMzMhJubm/jhhx+EEEJ4eHiI\nn3/++Zl9fvxzkZqaKgwMDERWVpZUdsuWLVJ/yvt2+fJlabuHh4eYN2+eyvZyI0eOFBs3blSJe/Dg\nwWLjxo3i3r17okOHDuLMmTPStk2bNqns/674+OOPVb7Tyj3vd1Q5Z2dn6XhPmDBBfPbZZ8LNzU0I\nIcS6devEZ599JoSo+rw8GU+3bt1EWFiYVK78GimPy8DAQGzbtk3avm7dOtGvX79K+/D999+LyZMn\nq7Tr6+srrXNychKrV6+WtiUnJ7/wMajuOCdXpho1agRHR0eEhITg4sWLSE5OxqVLl2BmZoa0tDR0\n7NhRpXz5bbi0tDQ4OztL69977z3MmDHjudv94IMPpP9/+OGH0NbWxpo1a5CUlISkpCSkpKTAwcHh\nmW316dMHCxcuREJCAho3boyzZ8+qjMbQP9e8eXMAj26tHzhwAFu2bJG2lZSUoFu3brh58yZu376t\ncr20bdsWkyZNAgCMGDEC+/btw7lz55Camoq4uDiUlZX9o7jMzc3x4MEDJCcnIzIyEhYWFrh27RrO\nnj0LTU1NaGpqomPHjti0aRNu3boFU1NTaV8hBIqKilRuhwNAYmIiOnTooDJKoVQq8ccff0jLLVq0\nUNmnbt26KCwsrBBfZmYmiouLMXz48Aq3N9PT05GRkYEPP/xQZWSoU6dOOHbs2MsdkGpIQ0ND5bgb\nGRlJ18rjx/F5jlWrVq1Qq1Ytaf3gwYMBQOW2r62tLdq3b48BAwagQ4cOsLe3x7Bhw1CjhurNyNTU\nVLRu3Rp169aV1pmamqK0tBSXL18GADRs2BC1a9eWttepU0eazlAVHR0dmJubIzw8HA8ePMAHH3wA\nOzs7nDhxAp9++ikiIiLwxRdfPLPPj1+HiYmJ0NPTU/neVCqVFdp+/A6Wrq5updcl8Oiz7OfnJ43s\nAY9Gt9PS0pCWloaysjKVqRCdOnV6Zr/p6crvOjk7OyM6OhqBgYGYPHkygEcj+926dQPw9POSnp6u\nUt+tW7dw7do1GBkZSetat26NevXqqZR7kevhyJEjKp/V0tJStG7dWtpe/l0OPPpuf/yzKAdMcmUq\nNzcXTk5OMDIygo2NDYYNG4ajR48iOjoa77333lP309J6/kui/Pbx4x7/wZ6QkICRI0eiR48esLS0\nxPjx46V5js9qq379+rC2tsbBgwfRuHFjKJVK6OvrP3ds9Gzl56q0tBRubm4YNGhQhe1VXStCCIwf\nPx75+fno168f7O3tUVxcLH3JvyxtbW1YWloiPDwcUVFRcHR0RG5uLs6cOYOSkhLY2NgAeJSIt23b\nFqtWrapQh66ursqypqZmpfE/7smE6WlKS0uhoaGBzZs3qyRLANCgQQNkZGRUqLuq4/i2evzzW/6L\njYaGhsrDflUdq4YNGz7Xa7SARwnm9u3bERERgT///BM7d+7Eli1bKrzxorJbzmVlZRBCSDFWdi6e\nPF9PY2Njg/DwcBQWFsLc3BxmZmbw9/dHbGws6tSpg3bt2iEhIaHKPj8+P1tTU7NC25XF8uQt5KfF\nW1paijlz5qBLly4q6+vUqYPs7OwK+8rxunyTbG1tsXHjRsTFxUFfXx9WVlbQ0NBAXFwcIiMjMWfO\nHABVn5fHlX+mnnVNPPl9VtX14OjoiIkTJ1baTmX7yu2a4JxcmTp06BDq16+PNWvWwMXFBebm5tIc\nyw8//BAJCQkq5UeMGIH9+/dX2FZWVoYePXrg3LlzeO+993D//n1pW0ZGRpUx7N69G1ZWVvDz88OI\nESNgZGQkjaZUFsfjbQGPXgF15MgRHDt27JW9XoUeefyHZuvWrZGVlYWWLVtK/zZv3ozjx49DV1cX\n9evXVzlPFy9ehJ2dHZKSkhAVFYWQkBC4u7vDzs4Oubm5T23nRZQnE9HR0bCwsICFhQXOnj2LkydP\nSqMjrVu3xpUrV1C/fn0p7oyMDKxYsaJCux999JHKg4wApLmUL6ply5bQ1NTErVu3pHZr166NhQsX\n4ubNm/joo4+Qnp6O/Px8aZ8n237bCSFUronY2Fjo6+tDT09PpVxVx+rGjRto1aoVMjIyVEaiFi9e\njIULF6rUEx0djTVr1sDKygozZ87Eb7/9hsLCQpU5ssCjayItLQ13796V1p07dw5aWlr417/+9Y/7\n3a1bN0RFReHs2bOwsLCAQqFAaWkp1q9fD1tb26f2uU6dOlKfH9euXTvcvXtXSkCBF7sun7zOW7du\njatXr6p8lletWoXz58+jdevW0NTUVHnYLD4+/mUOA/2/Tp06obS0FNu3b4e5ubl0h+Pnn39Go0aN\npBHXqs7L43R1ddGkSRPExcVJ6zIzM1Wu5xfRunVrXL58WaXdP/74A7/++iuAR9+Lj18PWVlZL91W\ndcUkV6b09PRw5coVnD59GpmZmQgKCsLBgwdRXFyMESNGSMlJ+cNoKSkpsLS0hIuLC/bs2YNdu3Yh\nIyMDPj4+EEKgY8eO6NSpE06dOoXTp08jMTERCxYsqPIVTfXr18elS5cQExODtLQ0+Pr6IjY2Vnoa\nuaq2AKBnz55IT09HREQE+vTp80aO27vi8d/ex40bh3379mHDhg3IzMxESEgI1q9fL93ScnFxwYoV\nKxAeHo6kpCT4+PjAzMwM9erVg6amJvbu3YsrV67gwIED0hO95ef4eUfInmRra4s///wTurq6aNy4\nMTp06ICCggJERkZKyYStrS2aN2+Or7/+GomJiYiKisI333yD2rVrV/jh379/f+Tn58PHxwfp6enY\ntm0b9u/f/9xJeO3atXHt2jVkZ2ejTp06GDp0KLy9vREREYHk5GTMmDEDmZmZaNGiBaytrdG8eXPM\nmTMHKSkpCAsLw/79+1/qOFRnCxcuxIULF3Dq1Cn4+/tj1KhRFco861jZ2tqiUaNGmDt3LlJTU3H4\n8GFs3bpV+kWmnI6ODgICArB9+3ZkZ2dj3759KCgogIGBgUo5GxsbtGzZEjNmzEBiYiL+/vtvKgBS\nFwAAEGBJREFUfPvttxg4cKDKFIaXpVAoUKNGDRw/flwlqdm/f78Uc2V9nj59utRn4H+fi1atWsHW\n1haenp64dOkSTp48iZUrVz53POW3luPi4lBUVIRx48YhJCQEu3fvRmZmJvz8/HDgwAG0bdsWdevW\nxaBBg/Dtt98iJiYG4eHh0ueVXo6Ghga6du2KnTt3wtzcHMCj6VaPXw/Ao+/YdevWVTgv7dq1q1Dn\n6NGjpYe0ExISMHv2bJWHMJ+lVq1aSExMRH5+PkaOHIkLFy7g+++/x+XLl/Hrr79i+fLl0vSY0aNH\nY/369Th48CASExPh5eVV6V2vtxmTXJnq27cvHBwc8MUXX2Do0KGIiIjArFmzkJKSgqZNmyIgIAA7\nduzAwIEDcfDgQaxZswaNGzeGhYUFvL29ERgYCEdHR1y6dAlr166FtrY2HB0d0bt3b3h4eMDd3R0D\nBw5E48aNpTaf/BC6uLhAqVTi008/xejRo3H16lVMmjRJGj2oqi3g0Q+L8tfmNGjQ4M0dvHfA4+fK\nxMQE3333HX755RfpDygsW7ZM+tJ2d3dHr1698OWXX2LUqFFo3rw55s+fD319fcybNw8//vgjBgwY\ngB9++AFz586FpqamNHL5eDsvMqrbtm1bNGrUCBYWFgAeTSUwNTWFoaGh9E7JGjVqYPXq1QCA4cOH\nY+rUqfj444/h5eVVob7atWtjzZo1iIqKgoODA3bv3g0HB4cqb809Hm+vXr1QVlaGAQMG4ObNm5g1\naxZsbGwwZcoUjBgxAtra2tLbGrS0tLB27VrcuXMHTk5O2Lp1a6UJ4Nuub9+++Oyzz/D1119j+PDh\ncHd3r7RcVcdKU1MTq1atwvXr1zFkyBAsWrQIs2bNQvfu3VXqUCgUWLRoEX766Sf069cPQUFB8PPz\nQ5s2bVTKPXlNfP311+jZs2eVb/x40bsN1tbWaNSoEZo2bQrg0feYpqamyuukqurzk236+Pigdu3a\nGD58OObPnw8nJ6fnvmXcvn17WFtbw9nZGcePH0e/fv0wbdo0+Pv7Y+DAgQgPD8fatWulUey5c+fC\n1NQUn376KTw9PTFmzJgX6rtcPO2cv8ydJ1tbW5SUlEjfl+XfWY8nuf369cOXX35Z4byUj/Q+3u6E\nCRPQu3dvTJkyBePGjYO9vT00NDSka+JZMY4ZMwZ+fn4ICAhA8+bNsXr1ahw/fhwDBw6Ev78/PD09\n0b9/fwCAg4MDJk+ejAULFmD06NGwtbXF+++//8LHoDrTEC871EL0Bjg7O2PYsGHSwyhELyMrKwu5\nubnSDyIAmD9/PgoKCrBo0SI1Rvb2yc7ORs+ePXH48GHp4UV6OQ8fPsSpU6dgZ2cnjaAdOHAAfn5+\nOHz4sJqjI3X466+/YGRkJP0yf/PmTdjY2PDz9pL44BlVS+Hh4Thz5gxSU1M5VUFmbty4UeU0hgYN\nGjz3Q2DPKz8/H+PHj4efnx86deqECxcuYM+ePVi2bNkrbeddIcexkbt370rTbCqjq6v7wu9SfZaa\nNWti9uzZcHZ2hpOTE65fv47AwED07dv3lbZDL66goEDlGZQn6ejovJIpME/aunUrNm3ahOnTpwMA\nVqxYAWNjYya4L4kjuVQteXp64siRI1iwYAF69+6t7nDoFTI2Nq70lU1CCGhoaLy2EYvQ0FD88MMP\nyMnJQbNmzeDm5gYnJ6dX3o7cyXUk19XVVfpjH5VZtGhRhTeQvApnz57F4sWLkZiYiDp16sDR0RFf\nfPGF7J5yf9sEBQVh2bJlT50eMGjQoNdyF+jatWuYP38+IiIiIIRA165d4eXlhSZNmrzytt4FTHKJ\niIiISHb44BkRERERyQ6TXCIiIiKSHSa5RERERCQ7THKJiIiISHaY5BIRERGR7DDJJSIiIiLZYZJL\nRPSKKRQK7Nq1S91hvDVKSkoQEhKi7jCISGb4nlwiolfsxo0b0NXVhba2trpDeSvs2rULnp6euHjx\norpDISIZ4Z/1JSJ6xRo2bKjuEN4qZWVl6g6BiGSI0xWISNaOHTsGJycnKJVKWFtbw9PTE3fv3pW2\nZ2Rk4PPPP4eFhQU6d+6Mr776Cjdv3pS279q1C46OjjAxMYG9vT1Wr14tJWXZ2dlQKBQICgqCra0t\nevXqhfz8fJXpCp6envD09MTixYthbW0NpVKJiRMn4vr161IbmZmZcHNzg6mpKezs7BASEoLevXs/\ndcrDzp07YWdnh+3bt6Nbt24wMzPDpEmTcO3aNanM1atX8eWXX8La2hpGRkaws7PDkiVLVOro3bs3\nFi5cCAsLC0yaNAkAcOjQIQwbNgympqYwNjbGkCFDcOLECWk/FxcXfPfdd5g+fTpMTU1ha2uLLVu2\n4OzZsxg0aBCUSiVGjBiBjIwMaZ/c3Fx8+eWXsLS0ROfOnfH555/j8uXLUhyzZ8+GEAKGhoaIjIwE\nAPz5558YMmQITExM0Lt3b6xYsQJFRUVSnQqFAitXroS9vT26deuGjIwMxMTEYNSoUTA1NYWVlRWm\nTJmCq1evPu+lQkRyI4iIZOrmzZuiU6dOYvPmzeLq1avi7NmzomfPnsLLy0sIIcTdu3eFra2tcHV1\nFfHx8eLixYti6NChwsXFRQghRHBwsLT/5cuXxZ49e4S5ubnw8fERQgiRlZUlDAwMRN++fUVycrK4\ncOGCEEIIAwMDsXPnTiGEELNmzRJGRkZi9uzZIjU1VURGRgobGxsxe/ZsIYQQBQUF4uOPPxaurq7i\n0qVLIioqSgwYMEAYGhpKdTwpLCxMdOzYUQwYMECcPXtWxMbGiiFDhogBAwaI0tJSIYQQjo6Ows3N\nTVy6dElkZmaKdevWCQMDA3Ho0CGpDgMDAzF16lSRmZkpxW9oaCjWr18vsrKyxMWLF4Wrq6uwtrYW\nxcXFQgghRo8eLYyMjERwcLDIzMwU3t7eomPHjsLBwUFERkaKCxcuCHt7ezF58mQhhBAPHjwQvXv3\nFtOmTROXLl0SSUlJYvbs2cLKykrk5uaKwsJCsW7dOqFQKMSNGzdEcXGxOHbsmDAxMRHbtm0TmZmZ\n4uTJk6JPnz7iiy++kI6BgYGB6Nq1q4iLixPnz58XpaWlomvXrmLFihUiOztbxMfHi6FDh4rx48e/\nuguKiN4qTHKJSLYuXrwoFAqFOHr0qLQuOTlZJCQkCCGE2Lx5szA1NRX37t2Ttl+6dEksW7ZMFBYW\nChsbG/Hdd9+p1Llu3TphZGQk7t27JyW5GzduVCnzZJJrbW0tSkpKpO0+Pj6iT58+QgghQkNDhVKp\nFHfv3pW2JyQkqNTxpLCwMKFQKER8fLy0LiUlRRgYGIgTJ06Ihw8fiuDgYJGTk6Oyn42NjVi1apVK\nHYmJiSrHa/PmzSr7HDt2TCgUCqmu0aNHi2HDhknbk5KShIGBgdi+fbu0zs/PT+rftm3bRNeuXaXk\nWwghysrKhL29vVi5cqVKLOVGjhwp/SJR7u+//xYGBgYiOztbCPHoGPv6+krb79y5IxQKhfjll19E\nWVmZEEKIzMxMcf78+UqPIRHJH+fkEpFsKRQK9O/fH5999hkaN24MGxsb/Pvf/0avXr0AAElJSWjV\nqhXq1q0r7dO+fXu0b98eN2/eRF5eHszMzFTqtLKyQklJCVJTU6W5t//617+qjKNly5bQ1NSUlnV1\ndVFcXAwAuHjxIlq3bg1dXV1pu4GBgcpyZerUqQNDQ0NpuU2bNqhXrx4SExNhY2ODkSNH4vfff8f5\n8+eRkZGBS5cu4caNGygtLVWp58MPP1Q5XvXq1cMPP/yA1NRUXL58WXoY7PH9Hu9vrVq1pD6W09HR\nkaYWXLx4Ebdv34a5ublKu8XFxUhLS6u0b/Hx8YiNjcW2bdtU1teoUQMpKSlo3rx5hdjff/99uLm5\nYf78+fj+++/RtWtX2NnZoW/fvk87hEQkc0xyiUjWlixZgkmTJuH48eM4deoUpk+fDgsLCwQHB0NL\n6+lfgeIpL54pKyuDEALvvfeetE5HR6fKGCp7y0J5/Zqamk9tqyqVxV5aWooaNWqgoKAAo0aNQlFR\nEfr06QMLCwsYGxtj5MiRVcYWEREBV1dX/Pvf/4a5uTkcHBzw4MEDab5uVW3XqFH5Ix5lZWVo06YN\nVq9eXWFb7dq1n7qPq6srBg8eXGFb48aNpf8/edynTZuGkSNH4tixYzh16hQWLFiAn376CTt37lQ5\nX0T0buCDZ0QkWzExMVi0aBFatWqFMWPGYM2aNfDx8cHff/+Nmzdvol27dkhPT0d+fr60T1xcHKyt\nrVFSUoJGjRrhzJkzKnVGRkZCW1tbZeTyn1AoFEhPT1d5GC4lJQX37t2rcr87d+4gKytLWk5KSkJ+\nfj46duyIEydO4OLFi1i/fj0mTZqEPn36oHbt2sjLy6uyzuDgYHTp0gX+/v4YO3YsunbtiitXrgB4\netL/LB999BGys7Ohq6uLli1bomXLlmjWrBn8/Pykh8w0NDQq7JOWliaVb9myJa5cuYLFixfj/v37\nlbaTlpaGefPmoUGDBhg+fDhWrFiBH3/8EcnJyUhISHip2Ino7cYkl4hkq06dOti0aROWLFmCjIwM\nJCYmYv/+/WjVqhUaNGiAgQMHQk9PDzNmzMClS5dw4cIFzJs3DwqFAvr6+pgwYQI2bdqEzZs3IyMj\nA7/++isCAwMxfPhwlSkO/0T//v1Rv359fPXVV0hISEB0dDSmT59eIfF7khAC06dPR1xcHKKjozFz\n5kyYmZnBwsIC+vr6AIDdu3fjypUriIqKgoeHB0pLS1XeUPCkZs2a4dKlSzhz5gyys7OxY8cO+Pv7\nA0CV+1XF0dERenp6mDx5MmJiYpCSkoKZM2fir7/+Qvv27QH8b0Q3Li4OhYWFcHNzw++//47AwECk\np6fj9OnT8PT0xP3795/6erb69etj3759+Oabb5CSkoK0tDSEhYWhXr16aNOmzUvFTkRvNya5RCRb\nbdu2RWBgIMLDwzFo0CCMGjUKWlpaCAoKAvDodvePP/6IkpISjBgxAu7u7vjoo4+wfPlyAMD48eMx\nY8YMrFu3Dv3798fKlSvh7u6O2bNnS21Ulow+K0F9nLa2tkoMU6dOxdChQyGEqPKPSWhoaMDBwQHu\n7u5wd3eHgYEB1q5dCwAwNjbGrFmzsGHDBvTr1w9z5syBlZUV+vfvj9jY2KfWOWXKFJiYmODzzz/H\n4MGDERoaikWLFkFHR6fK/arqb926dbFx40bUr18frq6uGDZsGK5du4aQkBAp+ezSpQuMjY3h7OyM\no0eP4pNPPsHy5ctx+PBhODg4YMaMGejevTtWrlz51Db19PTw448/Ijs7GyNGjMCQIUNw5coVhISE\noE6dOk+Nj4jki3/xjIhIjbKzs5Geng4bGxtp3bVr19C9e3ds2rSpwgNbwP/eLcu/EEZE9HQcySUi\nUqOHDx/C3d0dP//8M7KyshAfHw8vLy+0bt0aSqVS3eEREb21mOQSEalR27ZtsXz5cuzduxcDBw7E\np59+Cl1dXQQHB6u8doyIiF4MpysQERERkexwJJeIiIiIZIdJLhERERHJDpNcIiIiIpIdJrlERERE\nJDtMcomIiIhIdpjkEhEREZHsMMklIiIiItlhkktEREREsvN/PX7n2IdcplgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2d8eb452b70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.set_style('white')\n",
    "fig,ax = plt.subplots(figsize=(8,6))\n",
    "sns.boxplot(data=scores)\n",
    "plt.xlabel('scoring parameters')\n",
    "plt.ylabel('score')\n",
    "plt.title('Classification scores for tuned parameters');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating feature importances\n",
    "The individual contribution to the classification for each feature (i.e., feature importances) can be obtained from a Random Forest classifier. This gives a good idea of the classification power of individual features and helps understanding which type of feature engineering is the most promising.\n",
    "\n",
    "Caution should be taken when interpreting feature importances, as highly correlated variables will tend to dilute their classification power between themselves and will rank lower than uncorelated variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OOB estimate of accuracy for prospectivity classification using all features: 0.723065798988\n",
      "Feature ranking:\n",
      "1. feature 282 Marine_ratio_5_centered (0.031530)\n",
      "2. feature 293 dist_NM_up (0.030115)\n",
      "3. feature 288 Marine_ratio_75_centered (0.029691)\n",
      "4. feature 292 dist_M_down (0.027875)\n",
      "5. feature 284 Marine_ratio_15_centered (0.027651)\n",
      "6. feature 285 Marine_ratio_20_centered (0.026680)\n",
      "7. feature 290 Marine_ratio_200_centered (0.026423)\n",
      "8. feature 286 Marine_ratio_30_centered (0.022103)\n",
      "9. feature 5 NM_M (0.021090)\n",
      "10. feature 289 Marine_ratio_100_centered (0.018909)\n",
      "11. feature 287 Marine_ratio_50_centered (0.017506)\n",
      "12. feature 283 Marine_ratio_10_centered (0.016424)\n",
      "13. feature 171 PE_moving_av_20ft (0.014572)\n",
      "14. feature 249 GR_moving_min_5ft (0.012678)\n",
      "15. feature 291 dist_M_up (0.011630)\n",
      "16. feature 250 GR_moving_min_7ft (0.011605)\n",
      "17. feature 81 PE_cA_level_3 (0.010347)\n",
      "18. feature 254 ILD_log10_moving_min_3ft (0.009445)\n",
      "19. feature 272 PE_moving_min_10ft (0.009424)\n",
      "20. feature 294 dist_NM_down (0.008873)\n",
      "21. feature 219 ILD_log10_moving_max_3ft (0.008808)\n",
      "22. feature 241 PHIND_moving_max_4ft (0.008783)\n",
      "23. feature 224 ILD_log10_moving_max_15ft (0.008776)\n",
      "24. feature 248 GR_moving_min_4ft (0.008697)\n",
      "25. feature 0 GR (0.008360)\n",
      "26. feature 223 ILD_log10_moving_max_10ft (0.008254)\n",
      "27. feature 271 PE_moving_min_7ft (0.008216)\n",
      "28. feature 170 PE_moving_av_10ft (0.007934)\n",
      "29. feature 251 GR_moving_min_10ft (0.007851)\n",
      "30. feature 277 PHIND_moving_min_5ft (0.007727)\n",
      "31. feature 275 PHIND_moving_min_3ft (0.007675)\n",
      "32. feature 257 ILD_log10_moving_min_7ft (0.007360)\n",
      "33. feature 240 PHIND_moving_max_3ft (0.007212)\n",
      "34. feature 247 GR_moving_min_3ft (0.007172)\n",
      "35. feature 159 ILD_log10_moving_av_5ft (0.007125)\n",
      "36. feature 221 ILD_log10_moving_max_5ft (0.007063)\n",
      "37. feature 174 PHIND_moving_av_5ft (0.006821)\n",
      "38. feature 37 DeltaPHI_cA_level_3 (0.006339)\n",
      "39. feature 160 ILD_log10_moving_av_10ft (0.006312)\n",
      "40. feature 255 ILD_log10_moving_min_4ft (0.006159)\n",
      "41. feature 259 ILD_log10_moving_min_15ft (0.005927)\n",
      "42. feature 269 PE_moving_min_4ft (0.005768)\n",
      "43. feature 31 ILD_log10_cA_level_1 (0.005768)\n",
      "44. feature 239 PE_moving_max_20ft (0.005693)\n",
      "45. feature 152 GR_moving_av_1ft (0.005620)\n",
      "46. feature 1 ILD_log10 (0.005587)\n",
      "47. feature 3 PHIND (0.005504)\n",
      "48. feature 6 RELPOS (0.005443)\n",
      "49. feature 153 GR_moving_av_2ft (0.005402)\n",
      "50. feature 34 ILD_log10_cA_level_4 (0.005058)\n",
      "51. feature 238 PE_moving_max_15ft (0.005040)\n",
      "52. feature 157 ILD_log10_moving_av_1ft (0.004991)\n",
      "53. feature 43 PHIND_cA_level_1 (0.004968)\n",
      "54. feature 246 PHIND_moving_max_20ft (0.004958)\n",
      "55. feature 276 PHIND_moving_min_4ft (0.004852)\n",
      "56. feature 230 DeltaPHI_moving_max_10ft (0.004707)\n",
      "57. feature 173 PHIND_moving_av_2ft (0.004687)\n",
      "58. feature 39 PE_cA_level_1 (0.004664)\n",
      "59. feature 32 ILD_log10_cA_level_2 (0.004654)\n",
      "60. feature 242 PHIND_moving_max_5ft (0.004548)\n",
      "61. feature 172 PHIND_moving_av_1ft (0.004467)\n",
      "62. feature 41 PE_cA_level_3 (0.004452)\n",
      "63. feature 158 ILD_log10_moving_av_2ft (0.004395)\n",
      "64. feature 256 ILD_log10_moving_min_5ft (0.004373)\n",
      "65. feature 264 DeltaPHI_moving_min_7ft (0.004358)\n",
      "66. feature 222 ILD_log10_moving_max_7ft (0.004249)\n",
      "67. feature 165 DeltaPHI_moving_av_10ft (0.004245)\n",
      "68. feature 270 PE_moving_min_5ft (0.004242)\n",
      "69. feature 161 ILD_log10_moving_av_20ft (0.004107)\n",
      "70. feature 38 DeltaPHI_cA_level_4 (0.004102)\n",
      "71. feature 155 GR_moving_av_10ft (0.004062)\n",
      "72. feature 236 PE_moving_max_7ft (0.003970)\n",
      "73. feature 204 PE_moving_std_20ft (0.003910)\n",
      "74. feature 79 PE_cA_level_1 (0.003905)\n",
      "75. feature 83 PHIND_cA_level_1 (0.003900)\n",
      "76. feature 278 PHIND_moving_min_7ft (0.003852)\n",
      "77. feature 33 ILD_log10_cA_level_3 (0.003812)\n",
      "78. feature 220 ILD_log10_moving_max_4ft (0.003765)\n",
      "79. feature 169 PE_moving_av_5ft (0.003751)\n",
      "80. feature 86 PHIND_cA_level_4 (0.003729)\n",
      "81. feature 231 DeltaPHI_moving_max_15ft (0.003673)\n",
      "82. feature 176 PHIND_moving_av_20ft (0.003595)\n",
      "83. feature 243 PHIND_moving_max_7ft (0.003566)\n",
      "84. feature 70 GR_cA_level_4 (0.003537)\n",
      "85. feature 263 DeltaPHI_moving_min_5ft (0.003492)\n",
      "86. feature 212 GR_moving_max_3ft (0.003473)\n",
      "87. feature 229 DeltaPHI_moving_max_7ft (0.003400)\n",
      "88. feature 260 ILD_log10_moving_min_20ft (0.003384)\n",
      "89. feature 168 PE_moving_av_2ft (0.003371)\n",
      "90. feature 22 PE_cD_level_4 (0.003364)\n",
      "91. feature 245 PHIND_moving_max_15ft (0.003225)\n",
      "92. feature 274 PE_moving_min_20ft (0.003152)\n",
      "93. feature 82 PE_cA_level_4 (0.003132)\n",
      "94. feature 87 GR_entropy_foot2 (0.003090)\n",
      "95. feature 27 GR_cA_level_1 (0.003087)\n",
      "96. feature 10 GR_cD_level_4 (0.003044)\n",
      "97. feature 54 ILD_log10_cD_level_4 (0.003015)\n",
      "98. feature 40 PE_cA_level_2 (0.002986)\n",
      "99. feature 62 PE_cD_level_4 (0.002894)\n",
      "100. feature 2 DeltaPHI (0.002862)\n",
      "101. feature 98 ILD_log10_entropy_foot10 (0.002792)\n",
      "102. feature 78 DeltaPHI_cA_level_4 (0.002736)\n",
      "103. feature 266 DeltaPHI_moving_min_15ft (0.002735)\n",
      "104. feature 45 PHIND_cA_level_3 (0.002653)\n",
      "105. feature 261 DeltaPHI_moving_min_3ft (0.002625)\n",
      "106. feature 166 DeltaPHI_moving_av_20ft (0.002619)\n",
      "107. feature 46 PHIND_cA_level_4 (0.002609)\n",
      "108. feature 154 GR_moving_av_5ft (0.002592)\n",
      "109. feature 77 DeltaPHI_cA_level_3 (0.002551)\n",
      "110. feature 232 DeltaPHI_moving_max_20ft (0.002545)\n",
      "111. feature 252 GR_moving_min_15ft (0.002541)\n",
      "112. feature 268 PE_moving_min_3ft (0.002493)\n",
      "113. feature 9 GR_cD_level_3 (0.002458)\n",
      "114. feature 175 PHIND_moving_av_10ft (0.002419)\n",
      "115. feature 227 DeltaPHI_moving_max_4ft (0.002415)\n",
      "116. feature 30 GR_cA_level_4 (0.002377)\n",
      "117. feature 211 PHIND_moving_std_20ft (0.002376)\n",
      "118. feature 244 PHIND_moving_max_10ft (0.002362)\n",
      "119. feature 267 DeltaPHI_moving_min_20ft (0.002321)\n",
      "120. feature 163 DeltaPHI_moving_av_2ft (0.002276)\n",
      "121. feature 253 GR_moving_min_20ft (0.002248)\n",
      "122. feature 210 PHIND_moving_std_15ft (0.002224)\n",
      "123. feature 228 DeltaPHI_moving_max_5ft (0.002209)\n",
      "124. feature 92 GR_entropy_foot10 (0.002205)\n",
      "125. feature 279 PHIND_moving_min_10ft (0.002189)\n",
      "126. feature 68 GR_cA_level_2 (0.002179)\n",
      "127. feature 225 ILD_log10_moving_max_20ft (0.002164)\n",
      "128. feature 35 DeltaPHI_cA_level_1 (0.002157)\n",
      "129. feature 196 DeltaPHI_moving_std_15ft (0.002150)\n",
      "130. feature 235 PE_moving_max_5ft (0.002136)\n",
      "131. feature 237 PE_moving_max_10ft (0.002133)\n",
      "132. feature 233 PE_moving_max_3ft (0.002054)\n",
      "133. feature 17 DeltaPHI_cD_level_3 (0.002046)\n",
      "134. feature 44 PHIND_cA_level_2 (0.002030)\n",
      "135. feature 197 DeltaPHI_moving_std_20ft (0.002011)\n",
      "136. feature 74 ILD_log10_cA_level_4 (0.002005)\n",
      "137. feature 69 GR_cA_level_3 (0.001974)\n",
      "138. feature 84 PHIND_cA_level_2 (0.001971)\n",
      "139. feature 234 PE_moving_max_4ft (0.001941)\n",
      "140. feature 36 DeltaPHI_cA_level_2 (0.001940)\n",
      "141. feature 265 DeltaPHI_moving_min_10ft (0.001915)\n",
      "142. feature 25 PHIND_cD_level_3 (0.001901)\n",
      "143. feature 280 PHIND_moving_min_15ft (0.001895)\n",
      "144. feature 156 GR_moving_av_20ft (0.001888)\n",
      "145. feature 42 PE_cA_level_4 (0.001869)\n",
      "146. feature 226 DeltaPHI_moving_max_3ft (0.001856)\n",
      "147. feature 113 PHIND_entropy_foot4 (0.001846)\n",
      "148. feature 262 DeltaPHI_moving_min_4ft (0.001806)\n",
      "149. feature 58 DeltaPHI_cD_level_4 (0.001804)\n",
      "150. feature 66 PHIND_cD_level_4 (0.001798)\n",
      "151. feature 203 PE_moving_std_15ft (0.001797)\n",
      "152. feature 162 DeltaPHI_moving_av_1ft (0.001776)\n",
      "153. feature 164 DeltaPHI_moving_av_5ft (0.001759)\n",
      "154. feature 88 GR_entropy_foot3 (0.001689)\n",
      "155. feature 14 ILD_log10_cD_level_4 (0.001689)\n",
      "156. feature 18 DeltaPHI_cD_level_4 (0.001684)\n",
      "157. feature 90 GR_entropy_foot5 (0.001682)\n",
      "158. feature 89 GR_entropy_foot4 (0.001646)\n",
      "159. feature 215 GR_moving_max_7ft (0.001638)\n",
      "160. feature 217 GR_moving_max_15ft (0.001597)\n",
      "161. feature 73 ILD_log10_cA_level_3 (0.001554)\n",
      "162. feature 28 GR_cA_level_2 (0.001541)\n",
      "163. feature 216 GR_moving_max_10ft (0.001515)\n",
      "164. feature 218 GR_moving_max_20ft (0.001498)\n",
      "165. feature 273 PE_moving_min_15ft (0.001492)\n",
      "166. feature 190 ILD_log10_moving_std_20ft (0.001424)\n",
      "167. feature 189 ILD_log10_moving_std_15ft (0.001422)\n",
      "168. feature 50 GR_cD_level_4 (0.001421)\n",
      "169. feature 258 ILD_log10_moving_min_10ft (0.001419)\n",
      "170. feature 94 ILD_log10_entropy_foot3 (0.001407)\n",
      "171. feature 182 GR_moving_std_15ft (0.001398)\n",
      "172. feature 202 PE_moving_std_10ft (0.001389)\n",
      "173. feature 97 ILD_log10_entropy_foot7 (0.001343)\n",
      "174. feature 67 GR_cA_level_1 (0.001277)\n",
      "175. feature 65 PHIND_cD_level_3 (0.001261)\n",
      "176. feature 183 GR_moving_std_20ft (0.001236)\n",
      "177. feature 61 PE_cD_level_3 (0.001223)\n",
      "178. feature 96 ILD_log10_entropy_foot5 (0.001221)\n",
      "179. feature 12 ILD_log10_cD_level_2 (0.001213)\n",
      "180. feature 29 GR_cA_level_3 (0.001206)\n",
      "181. feature 209 PHIND_moving_std_10ft (0.001194)\n",
      "182. feature 115 PHIND_entropy_foot7 (0.001187)\n",
      "183. feature 213 GR_moving_max_4ft (0.001178)\n",
      "184. feature 57 DeltaPHI_cD_level_3 (0.001171)\n",
      "185. feature 95 ILD_log10_entropy_foot4 (0.001152)\n",
      "186. feature 281 PHIND_moving_min_20ft (0.001129)\n",
      "187. feature 181 GR_moving_std_10ft (0.001116)\n",
      "188. feature 8 GR_cD_level_2 (0.001071)\n",
      "189. feature 80 PE_cA_level_2 (0.001048)\n",
      "190. feature 26 PHIND_cD_level_4 (0.001042)\n",
      "191. feature 111 PHIND_entropy_foot2 (0.001025)\n",
      "192. feature 179 GR_moving_std_5ft (0.001022)\n",
      "193. feature 75 DeltaPHI_cA_level_1 (0.001017)\n",
      "194. feature 214 GR_moving_max_5ft (0.001001)\n",
      "195. feature 180 GR_moving_std_7ft (0.000997)\n",
      "196. feature 85 PHIND_cA_level_3 (0.000971)\n",
      "197. feature 114 PHIND_entropy_foot5 (0.000971)\n",
      "198. feature 201 PE_moving_std_7ft (0.000905)\n",
      "199. feature 72 ILD_log10_cA_level_2 (0.000866)\n",
      "200. feature 13 ILD_log10_cD_level_3 (0.000861)\n",
      "201. feature 186 ILD_log10_moving_std_5ft (0.000842)\n",
      "202. feature 194 DeltaPHI_moving_std_7ft (0.000805)\n",
      "203. feature 21 PE_cD_level_3 (0.000796)\n",
      "204. feature 93 ILD_log10_entropy_foot2 (0.000760)\n",
      "205. feature 60 PE_cD_level_2 (0.000742)\n",
      "206. feature 64 PHIND_cD_level_2 (0.000741)\n",
      "207. feature 200 PE_moving_std_5ft (0.000726)\n",
      "208. feature 139 PEgradient_dx3 (0.000719)\n",
      "209. feature 187 ILD_log10_moving_std_7ft (0.000715)\n",
      "210. feature 7 GR_cD_level_1 (0.000714)\n",
      "211. feature 24 PHIND_cD_level_2 (0.000688)\n",
      "212. feature 53 ILD_log10_cD_level_3 (0.000687)\n",
      "213. feature 206 PHIND_moving_std_4ft (0.000684)\n",
      "214. feature 112 PHIND_entropy_foot3 (0.000655)\n",
      "215. feature 4 PE (0.000629)\n",
      "216. feature 76 DeltaPHI_cA_level_2 (0.000592)\n",
      "217. feature 71 ILD_log10_cA_level_1 (0.000589)\n",
      "218. feature 55 DeltaPHI_cD_level_1 (0.000575)\n",
      "219. feature 100 DeltaPHI_entropy_foot3 (0.000571)\n",
      "220. feature 208 PHIND_moving_std_7ft (0.000568)\n",
      "221. feature 207 PHIND_moving_std_5ft (0.000560)\n",
      "222. feature 104 DeltaPHI_entropy_foot10 (0.000551)\n",
      "223. feature 195 DeltaPHI_moving_std_10ft (0.000547)\n",
      "224. feature 205 PHIND_moving_std_3ft (0.000535)\n",
      "225. feature 116 PHIND_entropy_foot10 (0.000525)\n",
      "226. feature 188 ILD_log10_moving_std_10ft (0.000519)\n",
      "227. feature 101 DeltaPHI_entropy_foot4 (0.000516)\n",
      "228. feature 192 DeltaPHI_moving_std_4ft (0.000513)\n",
      "229. feature 49 GR_cD_level_3 (0.000511)\n",
      "230. feature 128 ILD_log10gradient_dx6 (0.000503)\n",
      "231. feature 140 PEgradient_dx4 (0.000470)\n",
      "232. feature 56 DeltaPHI_cD_level_2 (0.000447)\n",
      "233. feature 19 PE_cD_level_1 (0.000443)\n",
      "234. feature 119 GRgradient_dx4 (0.000442)\n",
      "235. feature 122 GRgradient_dx10 (0.000438)\n",
      "236. feature 191 DeltaPHI_moving_std_3ft (0.000429)\n",
      "237. feature 198 PE_moving_std_3ft (0.000415)\n",
      "238. feature 138 PEgradient_dx2 (0.000411)\n",
      "239. feature 129 ILD_log10gradient_dx10 (0.000406)\n",
      "240. feature 193 DeltaPHI_moving_std_5ft (0.000394)\n",
      "241. feature 145 PHINDgradient_dx2 (0.000389)\n",
      "242. feature 125 ILD_log10gradient_dx3 (0.000387)\n",
      "243. feature 52 ILD_log10_cD_level_2 (0.000387)\n",
      "244. feature 151 PHINDgradient_dx20 (0.000368)\n",
      "245. feature 127 ILD_log10gradient_dx5 (0.000368)\n",
      "246. feature 121 GRgradient_dx6 (0.000347)\n",
      "247. feature 132 DeltaPHIgradient_dx3 (0.000344)\n",
      "248. feature 99 DeltaPHI_entropy_foot2 (0.000341)\n",
      "249. feature 51 ILD_log10_cD_level_1 (0.000336)\n",
      "250. feature 63 PHIND_cD_level_1 (0.000321)\n",
      "251. feature 146 PHINDgradient_dx3 (0.000315)\n",
      "252. feature 147 PHINDgradient_dx4 (0.000314)\n",
      "253. feature 141 PEgradient_dx5 (0.000312)\n",
      "254. feature 136 DeltaPHIgradient_dx10 (0.000311)\n",
      "255. feature 103 DeltaPHI_entropy_foot7 (0.000310)\n",
      "256. feature 167 PE_moving_av_1ft (0.000301)\n",
      "257. feature 126 ILD_log10gradient_dx4 (0.000293)\n",
      "258. feature 149 PHINDgradient_dx6 (0.000293)\n",
      "259. feature 48 GR_cD_level_2 (0.000289)\n",
      "260. feature 102 DeltaPHI_entropy_foot5 (0.000280)\n",
      "261. feature 118 GRgradient_dx3 (0.000279)\n",
      "262. feature 15 DeltaPHI_cD_level_1 (0.000274)\n",
      "263. feature 11 ILD_log10_cD_level_1 (0.000258)\n",
      "264. feature 148 PHINDgradient_dx5 (0.000247)\n",
      "265. feature 123 GRgradient_dx20 (0.000244)\n",
      "266. feature 144 PEgradient_dx20 (0.000236)\n",
      "267. feature 150 PHINDgradient_dx10 (0.000231)\n",
      "268. feature 124 ILD_log10gradient_dx2 (0.000230)\n",
      "269. feature 108 PE_entropy_foot5 (0.000223)\n",
      "270. feature 142 PEgradient_dx6 (0.000223)\n",
      "271. feature 185 ILD_log10_moving_std_4ft (0.000197)\n",
      "272. feature 16 DeltaPHI_cD_level_2 (0.000194)\n",
      "273. feature 135 DeltaPHIgradient_dx6 (0.000187)\n",
      "274. feature 178 GR_moving_std_4ft (0.000178)\n",
      "275. feature 20 PE_cD_level_2 (0.000170)\n",
      "276. feature 59 PE_cD_level_1 (0.000169)\n",
      "277. feature 130 ILD_log10gradient_dx20 (0.000166)\n",
      "278. feature 23 PHIND_cD_level_1 (0.000166)\n",
      "279. feature 137 DeltaPHIgradient_dx20 (0.000162)\n",
      "280. feature 134 DeltaPHIgradient_dx5 (0.000160)\n",
      "281. feature 133 DeltaPHIgradient_dx4 (0.000151)\n",
      "282. feature 143 PEgradient_dx10 (0.000150)\n",
      "283. feature 120 GRgradient_dx5 (0.000132)\n",
      "284. feature 110 PE_entropy_foot10 (0.000127)\n",
      "285. feature 106 PE_entropy_foot3 (0.000118)\n",
      "286. feature 47 GR_cD_level_1 (0.000105)\n",
      "287. feature 109 PE_entropy_foot7 (0.000104)\n",
      "288. feature 184 ILD_log10_moving_std_3ft (0.000100)\n",
      "289. feature 107 PE_entropy_foot4 (0.000093)\n",
      "290. feature 177 GR_moving_std_3ft (0.000077)\n",
      "291. feature 131 DeltaPHIgradient_dx2 (0.000072)\n",
      "292. feature 199 PE_moving_std_4ft (0.000071)\n",
      "293. feature 117 GRgradient_dx2 (0.000065)\n",
      "294. feature 91 GR_entropy_foot7 (0.000041)\n",
      "295. feature 105 PE_entropy_foot2 (0.000000)\n"
     ]
    }
   ],
   "source": [
    "####### Evaluation of feature importances\n",
    "Cl = RandomForestClassifier(n_estimators=75, max_features=0.1, min_samples_leaf=25,\n",
    "                            min_samples_split=50, class_weight='balanced', random_state=42,oob_score=True, n_jobs=-1)\n",
    "Cl.fit(X, y)\n",
    "print ('OOB estimate of accuracy for prospectivity classification using all features: %s' % str(Cl.oob_score_))\n",
    "\n",
    "importances = Cl.feature_importances_ \n",
    "std = np.std([tree.feature_importances_ for tree in Cl.estimators_], axis=0)\n",
    "indices = np.argsort(importances)[::-1]\n",
    "print(\"Feature ranking:\")\n",
    "Vars = list(X.columns.values)\n",
    "for f in range(X.shape[1]):\n",
    "    print(\"%d. feature %d %s (%f)\" % (f + 1, indices[f], Vars[indices[f]], importances[indices[f]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Plot the feature importances of the forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABOAAAAGzCAYAAACGmWiOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzs3Xt0V/WZL/53IEAQ8AZK1aVy4BwbLg6RmFgtVKfSWs+x\nI62CrtEKVcTfdFCnnQIiVqoyXCp1dapVW0exRYcealntsl6W1R6dM04PF7mWwBhiZ9p6gyi2ooEg\n5PeHJWOMWr40X0PC67VWluxnP/u7n+2f7/XZ+1PS1NTUFAAAAACgKLq09wAAAAAA0JkJ4AAAAACg\niARwAAAAAFBEAjgAAAAAKCIBHAAAAAAUkQAOAAAAAIpIAAcAAAAARSSAAwAAAIAiEsABAAAAQBHt\nFwFcY2Njrr322lRVVWXUqFFZsGDB+/bW1NRk3LhxqaioyNixY7N+/frmc7t37878+fMzcuTIVFZW\n5u/+7u/yyiuvNJ/fsGFDysvLM3jw4JSXl6e8vDznn39+UZ8NAAAAgAPbfhHAzZs3LzU1NVm4cGFm\nzpyZ2267LY899lirvoaGhkyaNClVVVVZsmRJKioqcsUVV2T79u1Jku9+97t55JFH8o//+I9ZvHhx\nfv/732fq1KnN12/atClDhgzJ008/3fx39913f2jPCQAAAMCBp90DuIaGhjzwwAO57rrrUl5entGj\nR2fixIm57777WvU+9NBD6dmzZ6ZMmZKBAwdmxowZ6dWrVx599NEkb6+Amz59eiorKzNo0KB84Qtf\nyMqVK5uvr6ury8CBA3P44Yenb9++6du3bw455JAP7VkBAAAAOPC0ewC3cePG7Nq1KxUVFc21ysrK\nrF27tlXv2rVrU1lZ2aI2YsSIrFq1Kknyt3/7txk9enSS5JVXXsmPfvSjnHLKKc29dXV1GTBgQBGe\nAgAAAADeW2l7D7Bly5YceuihKS39r1H69u2bHTt2ZOvWrTnssMOa65s3b84JJ5zQ4vq+fftm06ZN\nLWq33nprvvOd7+SQQw7JokWLmut1dXXZvXt3PvvZz2bbtm0ZNWpUpk6dmt69exfp6QAAAAA40LX7\nCriGhoZ07969RW3PcWNjY4v69u3b37P33X1jxozJj3/845x22mm59NJL88Ybb+Stt97Kb37zm+za\ntStz587N7Nmzs2rVqkybNq0ITwUAAAAAb2v3FXA9evRoFaDtOe7Zs+de9ZaVlbWoHXvssUne3tzh\nE5/4RH7+859nzJgxWbp0acrKytK1a9ckydy5c3Peeedly5YtOeKII/7krCeffHJ27NiRI488srCH\nBAAAAKBT2bx5c3r06JEVK1b8yd52XwHXv3//vPbaa9m9e3dzrb6+PmVlZTn44INb9W7ZsqVFrb6+\nvjk8e/LJJ7N58+bmc927d8+xxx6brVu3Jkl69erVHL4lyaBBg5IkL7/88l7N2tjYmF27dhXwdAAA\nAAB0Rrt27Wq1UOz9tPsKuMGDB6e0tDSrV6/OiBEjkiQrVqzIsGHDWvUOHz48d911V4vaypUr86Uv\nfSnJ2yvePve5z2XSpElJkm3btuU//uM/MmjQoNTV1WXs2LF58MEHc8wxxyRJampqUlpamuOPP36v\nZt0T9D3xxBP79rAAAAAAdApnnnnmXve2+wq4srKynHvuuZk5c2bWrVuXxx9/PAsWLMj48eOTvL3C\nbceOHUmSs846K6+//npmz56durq6zJo1Kw0NDfnMZz6TJLnoooty991356mnnkptbW2mTJmSAQMG\n5BOf+EQGDhyYAQMG5Gtf+1pqa2uzYsWKXH/99bngggvSp0+fdnt+AAAAADq3dg/gkmT69OkZNmxY\nxo8fn5tuuilXX311Ro8enSQZOXJkHnnkkSRJ7969c+edd2bFihU577zzsm7dutx1113N34C76KKL\nMnHixNxwww0ZN25cSktLc/vttydJSkpKcscdd6R37965+OKLM3ny5Jx22mm55ppr2uehAQAAADgg\nlDQ1NTW19xAdxZ6lhV5BBQAAADiwFZIT7Rcr4AAAAACgsxLAAQAAAEARCeAAAAAAoIgEcAAAAABQ\nRAI4AAAAACgiARwAAAAAFJEADgAAAACKSAAHAAAAAEUkgAMAAACAIhLAAQAAAEARCeAAAAAAoIgE\ncAAAAABQRAI4AAAAACgiARwAAAAAFJEADgAAAACKSAAHAAAAAEUkgAMAAACAIhLAAQAAAEARCeAA\nAAAAoIgEcAAAAABQRAI4AAAAACgiARwAAAAAFJEADgAAAACKSAAHAAAAAEUkgAMAAACAIhLAAQAA\nAEARCeAAAAAAoIgEcAAAAABQRAI4AAAAACgiARwAAAAAFJEADgAAAACKSAAHAAAAAEVU2t4DdESN\njY1Zs2ZN8/Hw4cPTvXv3dpwIAAAAgP2VAG4frFmzJuurqzM0yfokWbYsVVVV7TwVAAAAAPsjAdw+\nGppE5AYAAADAn+IbcAAAAABQRAI4AAAAACgiARwAAAAAFJEADgAAAACKSAAHAAAAAEUkgAMAAACA\nIhLAAQAAAEARCeAAAAAAoIj2iwCusbEx1157baqqqjJq1KgsWLDgfXtramoybty4VFRUZOzYsVm/\nfn3zud27d2f+/PkZOXJkKisr83d/93d55ZVXWlw/f/78nHrqqTnllFNy8803F+2ZAAAAACDZTwK4\nefPmpaamJgsXLszMmTNz22235bHHHmvV19DQkEmTJqWqqipLlixJRUVFrrjiimzfvj1J8t3vfjeP\nPPJI/vEf/zGLFy/O73//+0ydOrX5+nvuuScPP/xwbr/99tx666158MEHPzDsAwAAAIA/V7sHcA0N\nDXnggQdy3XXXpby8PKNHj87EiRNz3333tep96KGH0rNnz0yZMiUDBw7MjBkz0qtXrzz66KNJ3l4B\nN3369FRWVmbQoEH5whe+kJUrVzZfv3Dhwlx11VU56aSTUl1dna9+9avveR8AAAAAaCvtHsBt3Lgx\nu3btSkVFRXOtsrIya9eubdW7du3aVFZWtqiNGDEiq1atSpL87d/+bUaPHp0keeWVV/KjH/0op5xy\nSpJk8+bNefHFF3PyySe3uM8LL7yQ+vr6Nn8uAAAAAEj2gwBuy5YtOfTQQ1NaWtpc69u3b3bs2JGt\nW7e26N28eXOOPPLIFrW+ffvm5ZdfblG79dZb8/GPfzwrV65sfgV1y5YtKSkpaXF9v3790tTUlJde\neqmtHwsAAAAAkuwHAVxDQ0O6d+/eorbnuLGxsUV9+/bt79n77r4xY8bkxz/+cU477bRceumleeON\nN9LQ0NDitz/oPoXauXNnli9f3vz35/4eAAAAAJ1HuwdwPXr0aBVY7Tnu2bPnXvWWlZW1qB177LEZ\nOnRo5s2bl+3bt+fnP/95evTo0eK3P+g+haqtrc366uqkujrrq6uzZs2aP+v3AAAAAOg82j2A69+/\nf1577bXs3r27uVZfX5+ysrIcfPDBrXq3bNnSolZfX58jjjgiSfLkk09m8+bNzee6d++eY489Nlu3\nbk3//v3T1NTU4ntve15L3XP9n2Nokqo//hcAAAAA9mj3AG7w4MEpLS3N6tWrm2srVqzIsGHDWvUO\nHz68ecOFPVauXJmTTjopSTJv3rz85Cc/aT63bdu2/Md//EcGDRqUI488MkcffXSeeeaZFvc56qij\n0q9fv7Z+LAAAAABIsh8EcGVlZTn33HMzc+bMrFu3Lo8//ngWLFiQ8ePHJ3l7hduOHTuSJGeddVZe\nf/31zJ49O3V1dZk1a1YaGhrymc98Jkly0UUX5e67785TTz2V2traTJkyJQMGDMgnPvGJJMmFF16Y\n+fPnZ9myZVm6dGluueWW5vu0Jd+EAwAAAGCP0j/dUnzTp0/PDTfckPHjx6dPnz65+uqrM3r06CTJ\nyJEjM3fu3IwZMya9e/fOnXfemZkzZ2bx4sX56Ec/mrvuuqv5G3AXXXRRGhoacsMNN2Tr1q0ZOXJk\nbr/99ub7TJw4MVu3bs2VV16Zrl27ZuzYsUUJ4Gpra9M0YUKGJlmfJMuWpaqqqs3vAwAAAMD+r6Sp\nqampvYfoKM4888wkydy5c5Pq6lQlWZ6k5t57M2TChPc9FsABAAAAdC57cqInnnjiT/a2+yuoAAAA\nANCZCeAAAAAAoIgEcAAAAABQRAI4AAAAACgiARwAAAAAFJEADgAAAACKSAAHAAAAAEUkgAMAAACA\nIhLAAQAAAEARCeAAAAAAoIgEcAAAAABQRAI4AAAAACgiARwAAAAAFJEADgAAAACKSAAHAAAAAEUk\ngAMAAACAIhLAAQAAAEARlbb3AAeCnTt3Zvny5c3Hw4cPT/fu3dtxIgAAAAA+LAK4D0FtbW2aJkzI\n0CTrk2TZslRVVbXzVAAAAAB8GARwH5KhSURuAAAAAAceAVw78EoqAAAAwIFDANcOvJIKAAAAcOAQ\nwLUTr6QCAAAAHBi6tPcAAAAAANCZCeAAAAAAoIgEcAAAAABQRAI4AAAAACgiARwAAAAAFJEADgAA\nAACKSAAHAAAAAEUkgAMAAACAIipt7wFIdu7cmeXLlzcfDx8+PN27d2/HiQAAAABoKwK4/UBtbW2a\nJkzI0CTrk2TZslRVVbXzVAAAAAC0BQHcfmJoEpEbAAAAQOcjgNvP7ExSW1PTouaVVAAAAICOSwC3\nn6lN0jRhQvOxV1IBAAAAOjYB3H7I66gAAAAAnUeX9h4AAAAAADozARwAAAAAFJEADgAAAACKSAAH\nAAAAAEUkgAMAAACAIhLAAQAAAEARCeAAAAAAoIj2iwCusbEx1157baqqqjJq1KgsWLDgfXtramoy\nbty4VFRUZOzYsVm/fn2L89/73vdy5plnprKyMl/84hdTV1fXfG7Dhg0pLy/P4MGDU15envLy8px/\n/vlFey4AAAAA2C8CuHnz5qWmpiYLFy7MzJkzc9ttt+Wxxx5r1dfQ0JBJkyalqqoqS5YsSUVFRa64\n4ops3749SbJo0aLce++9uf7667NkyZIcc8wxufzyy7Njx44kyaZNmzJkyJA8/fTTzX933333h/qs\nAAAAABxY2j2Aa2hoyAMPPJDrrrsu5eXlGT16dCZOnJj77ruvVe9DDz2Unj17ZsqUKRk4cGBmzJiR\nXr165dFHH02S/OQnP8lll12W008/Pccff3y+/vWvZ+vWrVm5cmWSpK6uLgMHDszhhx+evn37pm/f\nvjnkkEM+1OcFAAAA4MDS7gHcxo0bs2vXrlRUVDTXKisrs3bt2la9a9euTWVlZYvaiBEjsmrVqiTJ\ntGnTcs455zSfKykpSZK8/vrrSd4O4AYMGNDWjwAAAAAA76u0vQfYsmVLDj300JSW/tcoffv2zY4d\nO7J169YcdthhzfXNmzfnhBNOaHF93759s2nTpiRvh3HvtHjx4uzatSsnn3xykrcDuN27d+ezn/1s\ntm3bllGjRmXq1Knp3bt3sR4PAAAAgANcu6+Aa2hoSPfu3VvU9hw3Nja2qG/fvv09e9/dlyRr1qzJ\nN77xjUycODGHH3543nrrrfzmN7/Jrl27Mnfu3MyePTurVq3KtGnT2viJAAAAAOC/tPsKuB49erQK\n0PYc9+zZc696y8rKWtRWrVqVSZMm5fTTT89VV12VJCktLc3SpUtTVlaWrl27Jknmzp2b8847L1u2\nbMkRRxzRps8FAAAAAMl+sAKuf//+ee2117J79+7mWn19fcrKynLwwQe36t2yZUuLWn19fYvwbOnS\npbn00ktz6qmn5pvf/GaL3l69ejWHb0kyaNCgJMnLL7/cZs8DAAAAAO/U7gHc4MGDU1pamtWrVzfX\nVqxYkWHDhrXqHT58ePOGC3usXLmyeQOHZ599Nl/60pdyxhln5Fvf+laLsK2uri4jRozI888/31yr\nqalJaWlpjj/++LZ+LAAAAABIsh8EcGVlZTn33HMzc+bMrFu3Lo8//ngWLFiQ8ePHJ3l7hduOHTuS\nJGeddVZef/31zJ49O3V1dZk1a1YaGhpy9tlnJ0muv/76HH300bnmmmvy6quvpr6+vvn6gQMHZsCA\nAfna176W2trarFixItdff30uuOCC9OnTp92eHwAAAIDOrd0DuCSZPn16hg0blvHjx+emm27K1Vdf\nndGjRydJRo4cmUceeSRJ0rt379x5551ZsWJFzjvvvKxbty533XVXysrKUl9fnzVr1mTTpk0544wz\nMmrUqOa/Rx55JCUlJbnjjjvSu3fvXHzxxZk8eXJOO+20XHPNNe356AAAAAB0cu2+CUPy9iq4OXPm\nZM6cOa3Obdy4scXxiSeemCVLlrTq69evXzZs2PCB9+nfv3++/e1v/3nDAgAAAEAB9osVcAAAAADQ\nWQngAAAAAKCIBHAAAAAAUEQCOAAAAAAoIgEcAAAAABSRAA4AAAAAikgABwAAAABFJIADAAAAgCIS\nwAEAAABAEQngAAAAAKCIBHAAAAAAUEQCOAAAAAAoIgEcAAAAABTRPgdwy5cvzw9/+MNs27YtmzZt\nyltvvdWWcwEAAABAp1Ba6AXbtm3LxIkTs3r16pSUlOTjH/945s+fn9/+9re555570r9//2LMCQAA\nAAAdUsEr4G655ZYkyc9//vOUlZUlSaZMmZLu3bvnG9/4RttOBwAAAAAdXMEB3P/5P/8nU6dOzbHH\nHttcGzRoUK6//vr88pe/bNPhAAAAAKCjKziAe/XVV3PEEUe0qh988MF5880322QoAAAAAOgsCg7g\nTjzxxDzyyCOt6vfff3+GDBnSJkMBAAAAQGdR8CYMX/nKV3LppZdm7dq1eeutt3LHHXekrq4u69ev\nz913312MGQEAAACgwyp4BdyIESPywx/+MAcddFCOP/74rF69Oh/5yEdy//3355RTTinGjAAAAADQ\nYRW8Ai5JjjnmmEyePDnHHXdckuSxxx7L8ccf36aDAQAAAEBnUPAKuPXr12f06NFZtGhRc23u3Lk5\n55xz8uyzz7bpcAAAAADQ0RUcwM2dOzef/OQn8+Uvf7m59thjj2XUqFGZO3dumw4HAAAAAB1dwQHc\nr371q3zpS19K9+7dm2ulpaWZNGlS1qxZ06bDAQAAAEBHV/A34Hr16pXf/va3OfbYY1vUN2/e3CKU\no23sTFJbU9OiNnz4cP+vAQAAADqIglfAnXXWWbnhhhvyy1/+Mm+88UbeeOON/L//9/9yww035FOf\n+lQxZjyg1SZpmjAhqa5OqquzvrraSkMAAACADqTgFXB///d/n9/85jf54he/mJKSkub6pz71qUyd\nOrVNh+NtQ5NUtfcQAAAAAOyTggO4gw46KHfddVeee+65PPvss+nWrVsGDRqUAQMGFGE8AAAAAOjY\nCg7g9hg4cGAGDhzYlrMAAAAAQKdTcAD33HPP5cYbb8zKlSuzc+fOVuc3bNjQJoMBAAAAQGdQcAA3\nc+bMvPLKK/nqV7+aPn36FGMmAAAAAOg0Cg7g1qxZk0WLFmXo0KHFmAcAAAAAOpUuhV5w2GGHpVu3\nbsWYBQAAAAA6nYIDuIsvvji33HJLtm3bVox5AAAAAKBTKfgV1H/7t3/LihUrUl1dnb59+6Z79+4t\nzj/xxBNtNhwAAAAAdHQFB3CVlZWprKwsxiwAAAAA0OkUHMBNnjy5GHOwl3Ymqa2paT4ePnx4q1WI\nAAAAAOw/Cg7gkmTjxo159tlns3v37iRJU1NTGhsbs27dusyaNatNB6Sl2iRNEyYkSdYnybJlqaqq\naseJAAAAAPggBQdwCxYsyLx585IkJSUlaWpqav73ySef3LbT8Z6GJhG5AQAAAHQMBe+Cev/99+fy\nyy/PmjVrcthhh+Wpp57KT3/60wwaNChnnnlmMWYEAAAAgA6r4ADupZdeytixY9OjR4+Ul5dn3bp1\n+ehHP5prrrkmDzzwQDFmBAAAAIAOq+AA7qCDDsquXbuSJMcdd1w2bdqUJBk0aFCef/75tp0OAAAA\nADq4ggO4ESNG5Hvf+14aGhoyZMiQ/OIXv8ju3bvzzDPPpFevXvs0RGNjY6699tpUVVVl1KhRWbBg\nwfv21tTUZNy4camoqMjYsWOzfv36Fue/973v5cwzz0xlZWW++MUvpq6ursX5+fPn59RTT80pp5yS\nm2++eZ/mBQAAAIC9VXAA95WvfCX/9//+39x///35X//rf6W+vj7V1dWZNm1aPve5z+3TEPPmzUtN\nTU0WLlyYmTNn5rbbbstjjz3Wqq+hoSGTJk1KVVVVlixZkoqKilxxxRXZvn17kmTRokW59957c/31\n12fJkiU55phjcvnll2fHjh1JknvuuScPP/xwbr/99tx666158MEHPzDsAwAAAIA/V8EB3AknnJDH\nH388n//859OrV68sXrw4kydPzje/+c38/d//fcEDNDQ05IEHHsh1112X8vLyjB49OhMnTsx9993X\nqvehhx5Kz549M2XKlAwcODAzZsxIr1698uijjyZJfvKTn+Syyy7L6aefnuOPPz5f//rXs3Xr1qxc\nuTJJsnDhwlx11VU56aSTUl1dna9+9avveR8AAAAAaCsFB3CXXHJJGhsbc/jhhydJ+vXrlwkTJqSq\nqipjxowpeICNGzdm165dqaioaK5VVlZm7dq1rXrXrl2bysrKFrURI0Zk1apVSZJp06blnHPOaT5X\nUlKSJHn99dezefPmvPjiizn55JNb3OeFF15IfX19wXMDAAAAwN4o3Zump556KuvWrUuSLF++PHfe\neWcOOuigFj3/+Z//uU+bMGzZsiWHHnpoSkv/a5S+fftmx44d2bp1aw477LDm+ubNm3PCCSe0uL5v\n377NG0GMGDGixbnFixdn165dqayszEsvvZSSkpIceeSRzef79euXpqamvPTSS+nXr1/BswMAAADA\nn7JXAdwxxxyTG2+8MU1NTWlqasrDDz+cLl3+a/FcSUlJDjrooEydOrXgARoaGtK9e/cWtT3HjY2N\nLerbt29/z9539yXJmjVr8o1vfCMTJ05M37598+tf/7rFb3/QfQAAAACgrexVAPff//t/zxNPPJEk\n+cu//MssWbKkxcq0P0ePHj1aBWB7jnv27LlXvWVlZS1qq1atyqRJk3L66afnqquuar52T/+7g7d3\n3wcAAAAA2krB34A7/PDD8/LLL7fZAP37989rr72W3bt3N9fq6+tTVlaWgw8+uFXvli1bWtTq6+tz\nxBFHNB8vXbo0l156aU499dR885vfbHHtnv49tmzZkpKSkhbXAwAAAEBbKjiA+93vftfq+29/jsGD\nB6e0tDSrV69urq1YsSLDhg1r1Tt8+PDmDRf2WLlyZfMGDs8++2y+9KUv5Ywzzsi3vvWtdO3atbnv\nyCOPzFFHHZVnnnmmxX2OOuoo338DAAAAoGgKDuAmTpyYGTNm5Mknn8xzzz2XF154ocVfocrKynLu\nuedm5syZWbduXR5//PEsWLAg48ePT/L2irUdO3YkSc4666y8/vrrmT17durq6jJr1qw0NDTk7LPP\nTpJcf/31Ofroo3PNNdfk1VdfTX19fYvrL7zwwsyfPz/Lli3L0qVLc8sttzTfBwAAAACKYa++AfdO\n3/rWt7Jr164sX748JSUlzfWmpqaUlJRkw4YNBQ8xffr03HDDDRk/fnz69OmTq6++OqNHj06SjBw5\nMnPnzs2YMWPSu3fv3HnnnZk5c2YWL16cj370o7nrrrtSVlaW+vr6rFmzJklyxhlntPj9OXPmZMyY\nMZk4cWK2bt2aK6+8Ml27ds3YsWMFcAAAAAAUVcEB3IIFC9p8iLKyssyZMydz5sxpdW7jxo0tjk88\n8cQsWbKkVV+/fv3+ZPjXpUuXTJs2LdOmTfvzBgYAAACAvVRwAFddXd3871dffTWlpaWtNksAAAAA\nAN5W8DfgkuQHP/hBRo4cmY9//OM55ZRTMmrUqNx7771tPBoAAAAAdHwFr4D74Q9/mJtvvjl//dd/\nnaqqqjQ1NWX58uW55ZZb0rt375x//vnFmBMAAAAAOqSCA7h7770306ZNy8UXX9xc+9SnPpXjjz8+\n3//+9wVwAAAAAPAOBb+C+sILL+QTn/hEq/qoUaPyn//5n20yFAAAAAB0FgUHcEcffXR+9atftaqv\nW7cu/fr1a5OhAAAAAKCzKPgV1AsvvDA33HBDXnvttYwYMSJJ8swzz+Tb3/52LrnkkjYfkA+2c+fO\nLF++vPl4+PDh6d69eztOBAAAAMA7FRzAXXLJJXn++ecze/bs7Nq1K01NTSktLc2FF16Yv/mbvynG\njHyA2traNE2YkKFJ1ifJsmWpqqpq56kAAAAA2KPgAK5Lly6ZMWNGrr766jz33HNJkoEDB6Z3795t\nPhx7Z2gSkRsAAADA/qngAC5Jtm3blocffjjPPvtsunTpkqFDh+Yzn/lMevTo0dbzAQAAAECHVnAA\nV1dXl/Hjx+eNN97If/tv/y27du3K4sWLc/vtt+f73/9+PvKRjxRjTgAAAADokAreBXXWrFkZPHhw\nnnzyySxZsiQ//elP84tf/CJHH310Zs2aVYwZAQAAAKDDKjiAW716daZOnZpDDjmkuXb44Ydn2rRp\n+bd/+7c2HQ4AAAAAOrqCA7h+/frlpZdealXftm1bDj300DYZCgAAAAA6i4IDuKlTp+aGG27I448/\nnj/84Q958803s3Tp0lx//fW55JJL8sILLzT/AQAAAMCBruBNGK688sokyeTJk1NSUtJcb2pqyrx5\n8zJv3rw0NTWlpKQkGzZsaLtJAQAAAKADKjiA+8EPflCMOQAAAACgUyo4gKuuri7GHAAAAADQKRUc\nwL366qu56667Ultbm8bGxlbnrZADAAAAgP9ScAA3derUrFu3LqeddlrKysqKMRMAAAAAdBoFB3DP\nPPNMvvvd73oVFQAAAAD2QsEBXP/+/dOrV69izEIb2LlzZ5YvX958PHz48HTv3r0dJwIAAAA4sBUc\nwE2ZMiU33HBDvvzlL+fYY49Nly5dWpw/+uij22w4CldbW5umCRMyNMn6JFm2LFVVVe08FQAAAMCB\nq+AArqmpKXV1dbn00ktb1UtKSrJhw4Y2G459MzSJyA0AAABg/1BwADd79ux87GMfy7hx49KzZ89i\nzAQAAACYm9BuAAAgAElEQVQAnUbBAdyrr76aa665Jscee2wx5gEAAACATqXLn25p6ZRTTsmqVauK\nMQsAAAAAdDoFr4A7+eSTM3PmzDz55JM57rjjUlra8icmT57cZsMBAAAAQEdXcAC3aNGiHHbYYVm9\nenVWr17d4lxJSYkADgAAAADeoeAA7he/+EUx5gAAAACATmmvArgXXnghRx11VEpKSvLCCy98YO/R\nRx/dJoMBAAAAQGewVwHcmWeemX/9139N375988lPfjIlJSWtepqamlJSUpINGza0+ZAAAAAA0FHt\nVQD3/e9/P4ccckiS5Ac/+EFRBwIAAACAzmSvArjq6ur3/Df7v507d2b58uXNx8OHD0/37t3bcSIA\nAACAA0vBmzDQsdTW1qZpwoQMTbI+SZYtS1VVVTtPBQAAAHDgEMAdAIYmEbkBAAAAtI8u7T0AAAAA\nAHRmAjgAAAAAKKJ9CuA2btyY6dOn58ILL8zLL7+c+++/P0uXLm3r2QAAAACgwys4gPvVr36VcePG\n5Xe/+11+9atfpbGxMRs2bMhll12Wp556qhgzAgAAAECHVXAAN3/+/Hzxi1/MwoUL061btyTJrFmz\nctFFF+XWW29t8wEBAAAAoCPbpxVwY8aMaVW/6KKLUldX1yZDAQAAAEBnUXAA161bt2zbtq1V/cUX\nX0zPnj3bZCgAAAAA6CwKDuBGjx6db33rW/nDH/7QXKurq8s//MM/5IwzztinIRobG3Pttdemqqoq\no0aNyoIFC963t6amJuPGjUtFRUXGjh2b9evXv2ffHXfckenTp7eobdiwIeXl5Rk8eHDKy8tTXl6e\n888/f59mBgAAAIC9UXAAN23atLzxxhv52Mc+loaGhnz+85/POeeck65du2bq1Kn7NMS8efNSU1OT\nhQsXZubMmbntttvy2GOPtepraGjIpEmTUlVVlSVLlqSioiJXXHFFtm/f3qLvZz/7WW677bZW12/a\ntClDhgzJ008/3fx3991379PMAAAAALA3Sgu9oKSkJD/84Q/zy1/+MjU1Ndm9e3dOOOGEjBo1Kl26\nFJznpaGhIQ888EDuvvvu5lVpEydOzH333ZdPf/rTLXofeuih9OzZM1OmTEmSzJgxI//yL/+SRx99\nNGPGjMmuXbty44035qc//WmOO+64Vveqq6vLwIEDc/jhhxc8JwAAAADsi4ITszFjxmT9+vU59dRT\nc9lll+Xyyy/P6aefvk/hW5Js3Lgxu3btSkVFRXOtsrIya9eubdW7du3aVFZWtqiNGDEiq1atSpK8\n+eabqa2tzeLFi1v83h51dXUZMGDAPs0JAAAAAPui4BVwDQ0NKSsra7MBtmzZkkMPPTSlpf81St++\nfbNjx45s3bo1hx12WHN98+bNOeGEE1pc37dv32zatClJ0qdPn/zzP//z+96rrq4uu3fvzmc/+9ls\n27Yto0aNytSpU9O7d+82ex4AAAAAeKeCA7hLLrkkV155ZS666KIcd9xxrcK4qqqqgn6voaEh3bt3\nb1Hbc9zY2Niivn379vfsfXffe3nrrbfym9/8Jscdd1zmzp2bP/zhD5k9e3amTZuW73znOwXNDAAA\nAAB7q+AA7pZbbkmS3HTTTa3OlZSUZMOGDQX9Xo8ePVoFaHuOe/bsuVe9e7Mir7S0NEuXLk1ZWVm6\ndu2aJJk7d27OO++8bNmyJUcccURBcwMAAADA3ig4gHviiSfadID+/fvntddey+7du5u/I1dfX5+y\nsrIcfPDBrXq3bNnSolZfX7/X4VmvXr1aHA8aNChJ8vLLLwvgAAAAACiKgndOOOaYYz7wr1CDBw9O\naWlpVq9e3VxbsWJFhg0b1qp3+PDhzRsu7LFy5cr33HDh3erq6jJixIg8//zzzbWampqUlpbm+OOP\nL3huAAAAANgb+/QNuA/ygx/8oKDfKysry7nnnpuZM2dm9uzZefnll7NgwYLMnTs3ydsr3Pr06ZMe\nPXrkrLPOyi233JLZs2fnggsuyKJFi9LQ0JCzzz77T95n4MCBGTBgQL72ta9l+vTp+f3vf5+vf/3r\nueCCC9KnT5+CZgYAAACAvfVnr4Dr379/tm/fnrVr1+akk07apyGmT5+eYcOGZfz48bnpppty9dVX\nZ/To0UmSkSNH5pFHHkmS9O7dO3feeWdWrFiR8847L+vWrctdd921V9+AKykpyR133JHevXvn4osv\nzuTJk3Paaaflmmuu2aeZAQAAAGBvFLwCbs6cOe9Z/853vpOXXnppn4YoKyvLnDlz3vO3N27c2OL4\nxBNPzJIlS/Zpzv79++fb3/72Ps3YWezcuTPLly9vPh4+fHirnWUBAAAAaDsFr4B7P+eee27zSjX2\nX7W1tVlfXZ1UV2d9dXXWrFnT3iMBAAAAdGoFr4B7P6tWrUrXrl3b6ucooqFJqtp7CAAAAIADRJts\nwrBt27b8+7//e/76r/+6TYYCAAAAgM6i4ADu6KOPTklJSYtat27dcvHFF+ev/uqv2mwwAAAAAOgM\nCg7grrrqqnzkIx9Jly4tPx/31ltvpaamJn/xF3/RZsMBAAAAQEdX8CYMZ555Zl577bVW9d/97nf5\nwhe+0CZDAQAAAEBnsVcr4O6///7cc889SZKmpqacd955rVbA/eEPf8jRRx/d9hMCAAAAQAe2VwHc\n5z//+WzdujVNTU35zne+k8985jPp1atXi55evXrl05/+dFGGBAAAAICOaq8CuJ49e2by5MlJkpKS\nklx22WXp2bNnUQcDAAAAgM6g4E0YJk+enLfeeisvv/xydu3aleTt11IbGxuzbt06O6ECAAAAwDsU\nHMD967/+a6ZNm5ZXX3211bmysjIBHAAAAAC8Q8G7oN5yyy0ZMmRIvvvd76asrCy33XZbrr322vTu\n3Ts333xzMWYEAAAAgA6r4BVwmzZtyuzZs1NeXp7BgwfnoIMOyhe+8IUcdNBBufvuuzN69OhizAkA\nAAAAHVLBK+C6du2aPn36JEmOP/74PPvss0mSj33sY6mrq2vb6QAAAACggys4gPsf/+N/5Be/+EWS\nZODAgXnmmWeSJC+99FLbTgYAAAAAnUDBr6BOmjQpV111Vbp165Zzzjknt956ayZNmpR///d/z8c+\n9rFizAgAAAAAHVbBK+BGjx6dH/3oR6moqMhRRx2Vf/qnf0rXrl1z5pln5sYbbyzGjAAAAADQYRW8\nAi5Jhg4dmiRpbGxMdXV1qqur23QoAAAAAOgsCl4BlySLFi3KJz/5yVRUVOS3v/1tvv71r+f2229v\n69kAAAAAoMMrOIB78MEH881vfjOf+9zn0q1btyRvb8Zw55135p577mnzAQEAAACgIys4gLvnnnsy\nY8aMXHnllenS5e3LL7nkklx//fX53//7f7f5gBTXzp07s3z58ua/xsbG9h4JAAAAoFMpOID79a9/\nnZNPPrlV/ZRTTsmLL77YJkPx4amtrc366uqkujrrq6uzZs2a9h4JAAAAoFMpOIDr169ffv3rX7eq\nr1q1KkceeWSbDMWHa2iSqj/+FwAAAIC2VXAAd8EFF+TGG2/ME088kSR57rnnsmjRovzDP/xDPv/5\nz7f5gAAAAADQkZUWesHll1+e119/PV/5yleyY8eOXHHFFSktLc2FF16Y/+//+/+KMSMAAAAAdFgF\nB3BJ8pWvfCV/8zd/k02bNqWpqSkDBw5M796923o2AAAAAOjw9uoV1G984xt58803W9R69uyZE088\nMX/xF38hfAMAAACA97FXAdyCBQvS0NDQojZp0qRs3ry5KEMBAAAAQGexVwFcU1NTq9ry5cuzY8eO\nNh8IAAAAADqTgndBBQAAAAD2ngAOAAAAAIpor3dBLSkpKeYc7Cd27tyZ5cuXNx8PHz483bt3b8eJ\nAAAAADq2vQ7gZs2alR49ejQf79y5MzfffHN69erVom/OnDltNx0futra2jRNmJChSdYn2fn00+nW\nrVvzeYEcAAAAQGH2KoCrqqrKli1bWtROOumkbN26NVu3bi3KYLSfoUmq/vjvmncFclm2LFVVVe97\nLQAAAAAt7VUAt3DhwmLPwX7snYEcAAAAAIWxCQMAAAAAFJEADgAAAACKSAAHAAAAAEUkgAMAAACA\nIhLAAQAAAEARCeAAAAAAoIgEcAAAAABQRAI4AAAAACgiARwAAAAAFNF+EcA1Njbm2muvTVVVVUaN\nGpUFCxa8b29NTU3GjRuXioqKjB07NuvXr3/PvjvuuCPTp09vVZ8/f35OPfXUnHLKKbn55pvb7BkA\nAAAA4L3sFwHcvHnzUlNTk4ULF2bmzJm57bbb8thjj7Xqa2hoyKRJk1JVVZUlS5akoqIiV1xxRbZv\n396i72c/+1luu+22Vtffc889efjhh3P77bfn1ltvzYMPPviBYR8AAAAA/LnaPYBraGjIAw88kOuu\nuy7l5eUZPXp0Jk6cmPvuu69V70MPPZSePXtmypQpGThwYGbMmJFevXrl0UcfTZLs2rUrM2fOzHXX\nXZfjjjuu1fULFy7MVVddlZNOOinV1dX56le/+p734f3t3Lkzy5cvb/5rbGxs75EAAAAA9mvtHsBt\n3Lgxu3btSkVFRXOtsrIya9eubdW7du3aVFZWtqiNGDEiq1atSpK8+eabqa2tzeLFi1v8XpJs3rw5\nL774Yk4++eQW93nhhRdSX1/flo/UqdXW1mZ9dXVSXZ311dVZs2ZNe48EAAAAsF9r9wBuy5YtOfTQ\nQ1NaWtpc69u3b3bs2JGtW7e26N28eXOOPPLIFrW+ffvm5ZdfTpL06dMn//zP/5wTTjjhPe9TUlLS\n4vp+/fqlqakpL730Uls+Uqc3NEnVH/8LAAAAwAdr9wCuoaEh3bt3b1Hbc/zu1xu3b9/+nr178xpk\nQ0NDi9/+oPsAAAAAQFtp9wCuR48erQKwPcc9e/bcq96ysrK9us87f/uD7gMAAAAAbaXdA7j+/fvn\ntddey+7du5tr9fX1KSsry8EHH9yqd8uWLS1q9fX1OeKII/bqPnv699jzWureXA8AAAAA+6LdA7jB\ngwentLQ0q1evbq6tWLEiw4YNa9U7fPjw5g0X9li5cmWrDRfey5FHHpmjjjoqzzzzTIv7HHXUUenX\nr9+f8QQAAAAA8P7aPYArKyvLueeem5kzZ2bdunV5/PHHs2DBgowfPz7J2yvWduzYkSQ566yz8vrr\nr2f27Nmpq6vLrFmz0tDQkLPPPnuv7nXhhRdm/vz5WbZsWZYuXZpbbrml+T4AAAAAUAztHsAlyfTp\n0zNs2LCMHz8+N910U66++uqMHj06STJy5Mg88sgjSZLevXvnzjvvzIoVK3Leeedl3bp1ueuuu/bq\nG3BJMnHixPzP//k/c+WVV+bLX/5yPve5zwngAAAAACiq0vYeIHl7FdycOXMyZ86cVuc2btzY4vjE\nE0/MkiVL/uRvvtdvdenSJdOmTcu0adP2fVha2LlzZ5YvX958PHz48FY71QIAAAAcyPaLAI6Oq7a2\nNk0TJmRokvVJdj79dLp169Z8XiAHAAAAHOgEcPzZhiap+uO/a94VyGXZslRVVb3vtQAAAACdnQCO\nNvfOQA4AAADgQLdfbMIAAAAAAJ2VAA4AAAAAikgABwAAAABF5BtwFNXOnTuzfPny5mO7ogIAAAAH\nGgEcRVX7jl1R1ySpuffeDBkyJMnb4VySdOvW7T2PhXUAAABAZyCAo+j27Ipak6RpwoTm+qNJBv7x\n/LuP1yfZ+fTTzWFcIpADAAAAOiYBHB+qPWFc8nYgN+SDjt+xem59kixblqqqqgAAAAB0JAI49mvv\nDOwAAAAAOiK7oAIAAABAEVkBR4dhR1UAAACgIxLA0WHUvuubcO/epGHw4MHZsGFD87GADgAAANgf\nCODoUFps4vCuQK7m3ntt2gAAAADsdwRwdGjv3lXVpg0AAADA/kYAR6flm3EAAADA/kAAR6f1p74Z\nJ5ADAAAAPgwCODq1D/pmnG/EAQAAAB8GARwHlHcGcu9+RdUuqgAAAEAxCOA4YL37FVW7qAIAAADF\nIIDjgGYXVQAAAKDYurT3AAAAAADQmVkBB+/j3d+I8004AAAAYF8I4OB9vPsbcTuffjrdunVrPi+Q\nAwAAAPaGAA4+QItvxH1AILdz584kEdABAAAArQjgoADvF8g9mmTgH88nyZq8vavqkCFDkgjjAAAA\n4EAmgIM/w55AribJkLTcUbVpwoQkb6+Wy7JlqaqyvyoAAAAciARwUCTvXC0HAAAAHLgEcPAhsKMq\nAAAAHLgEcPAhePeOql5JBQAAgAOHAA4+JF5JBQAAgANTl/YeAAAAAAA6MwEcAAAAABSRV1ChHbx7\nU4bBgwdnw4YNzcc2aQAAAIDOQwAH7eDdmzLU3HuvTRoAAACgkxLAQTt556YMNe86fvcKOSviAAAA\noOMSwMF+6N0r5KyIAwAAgI5LAAf7KSviAAAAoHMQwEEHYEUcAAAAdFwCOOggPmhFnF1UAQAAYP8l\ngIMO6E/torrz6afTrVu35n6BHAAAALQfARx0UB+0i2rNuwK6dwdyVswBAADAh6dLew+QJI2Njbn2\n2mtTVVWVUaNGZcGCBe/bW1NTk3HjxqWioiJjx47N+vXrW5z/2c9+lk996lOpqKjI5MmTs3Xr1uZz\nGzZsSHl5eQYPHpzy8vKUl5fn/PPPL9pzQXvaE8gNzdsr5tZXVyfV1VlfXZ0f//jHLY5XrFiR5cuX\nN/81Nja28/QAAADQeewXK+DmzZuXmpqaLFy4ML/73e8ybdq0HHPMMfn0pz/doq+hoSGTJk3Kueee\nm7lz52bRokW54oor8vjjj6esrCxr167NddddlxtvvDHl5eW56aabMn369Nx5551Jkk2bNmXIkCH5\np3/6pzQ1NSVJSkv3i/8FUHR7u2JuTd5+pXXIkCFJ3v7eXBKvtAIAAMA+avf0qaGhIQ888EDuvvvu\n5lVpEydOzH333dcqgHvooYfSs2fPTJkyJUkyY8aM/Mu//EseffTRjBkzJvfff3/OPvvs/NVf/VWS\n5Oabb85f/uVf5vnnn88xxxyTurq6DBw4MIcffviH/pywv9sTyNUkaZowobn+aJKBfzyf2IUVAAAA\nCtXur6Bu3Lgxu3btSkVFRXOtsrIya9eubdW7du3aVFZWtqiNGDEiq1atSpKsXr26RSjwkY98JEcd\ndVTWrFmTJKmrq8uAAQOK8BTQuewJ46qS/P/t3XtQ1WUex/GPeBQSMQVEkcVddU28cVEybcnCTVMm\nlWrzmhfMInedjB1MzeLiDSzGRNNSdF3WGjZLsayWVveuu5l4Q0PUdDYVUKBZbUmEw2X/cPnlAeUi\n53iO8H7NOPH8nh+/83uO/zifnu/z7VVj3L+O3wMAAAAAALXZPYArLCxUx44dLUpBPTw8VFpaanF+\nmyQVFBTIy8vL4pqHh4cuXbpkPKvmvKenpy5evCjpegB34sQJjR07VqGhoYqJiVFxcbEtlgUAAAAA\nAABIcpAS1JpnSVWPax4Ef+3atZveW31fXfPl5eU6d+6cunfvrsTERH333XdasWKFFixYoHXr1ll7\nWUCzZZZ0OjvbGNNRFQAAAACAutk9gHN2dq4VtFWP77nnngbd6+LiUu+8yWTS/v375eLiotatW0uS\nEhMT9dRTT6mwsFCdO3e26rqA5uq0fjgj7itdb9hQ3cDhK0nmffssGjbUDOgI7AAAAAAALY3dA7gu\nXbro8uXLqqyslJPT9YrYoqIiubi4qEOHDrXuLSwstLhWVFRkhGdeXl4qKiqqNV9dlurq6mox16tX\nL0nSpUuXCOCARmhoR9WbBXT1BXYEcgAAAACA5sbuZ8D17dtXJpNJR44cMa5lZmZqwIABte4NCAgw\nGi5UO3TokIKCgiRJgYGBOnjwoDGXn5+vixcvKiAgQGfOnNGgQYOUm5trzGdnZ8tkMunHP/6xtZcF\ntGjVgVz/BoxPnz6tr4YMkYYM0VdDhhhNUwAAAAAAaC7svgPOxcVF48ePV2xsrFasWKFLly5py5Yt\nSkxMlHR9B5ubm5ucnZ312GOPadWqVVqxYoUmTpyotLQ0lZSUaPTo0ZKkyZMna/r06QoICNCAAQO0\nYsUKhYaGysfHR1VVVfrJT36i1157TYsWLdKVK1cUFxeniRMnys3NzZ5fAdDi3biDzmw268CBA8Yc\nJawAAAAAgLud3QM4SVq0aJHi4+M1Y8YMubm5ad68eXr00UclSSEhIUpMTFR4eLjat2+vd955R7Gx\nsdq2bZv69OmjlJQU4wy4wMBALVmyRMnJybpy5YpCQkK0dOlSSVKrVq309ttva/ny5XrmmWfUqlUr\njRs3TvPnz7fbugHUdpoSVgAAAABAM+MQAZyLi4sSEhKUkJBQay4nJ8diPHDgQO3YseOWzwoPD1d4\nePhN57p06aI1a9Y07WUB2FydZ8zVHNcI7AjkAAAAAACOxiECOABoioYGcmazWZJuOZYI7AAAAAAA\n1kcAB6DZuVUglyGpp35oBlFzfFTXS1z79esnifPmAAAAAADWQQAHoNmrDuSyJfWTZTlrzXHVzJmS\nGnbeHA0iAAAAAAANQQAHADdozHlz9TWI0Jdf6v777xcAAAAAoGUjgAOARmhMgwiz2awDBw4Yv8uO\nOQAAAABomQjgAMBGTjdyxxwdXAEAAACgeSKAAwAbasyOubo6uEp176CjoysAAAAAOC4COABwILd7\n5lx9HV1rBnSEcwAAAABw5xDAAYADa+gOuro6ukqWAR0dXQEAAADgziKAA4BmqmZYZxHQcT4dAAAA\nANwxBHAA0ELdqfPpJAI7AAAAAC0bARwAoEFu93y6hgR2BHQAAAAAmjMCOADAbWnKDrrGBHR0eAUA\nAABwtyOAAwDcEbcb0NXX4VUikAMAAADg2AjgAAAO4XY7vNYM5ChvBQAAAOBoCOAAAHedmmFddSB3\nO+fPEdgBAAAAsDUCOADAXa8p58/RMAIAAACArRHAAQCavTvVMEIioAMAAABQGwEcAAA1WDOgI5AD\nAAAAQAAHAEAjNSaguzGQM5vNkkRABwAAALQwBHAAAFjZrQK5DEk9/z8v1e7gWjOgq69hBOWvAAAA\nwN2BAA4AABurDuSyJfXTzTu4SpYBXUMaRjS14ysBHgAAAHBnEMABAGBHNctZawZ01mwgYe0Aj8AO\nAAAAaBgCOAAAmpFGBXY1x40M8Oo6366+MeEdAAAAWhICOAAAYLjdwK7m+XZ1jdltBwAAgJaGAA4A\nANy2us63q3PchN12UuPPt2vKmO61AAAAaCoCOAAAYBd38ny7powb272WcA4AAAA1EcABAIC7QpPO\nt2vCuLHda629W49ADwAA4O5HAAcAANBIdXavtfJuPVuW31JeCwAAcGcQwAEAAFiZVXfj2bD8trHl\ntdY+a4+wDwAAtBQEcAAAAA7OUcprrX3WHuW6AACgpSCAAwAAgKHO8tqbzDvK7r76du/RLAMAANgT\nARwAAADsxpq7+W61e6/m2Bq77wjwAABAYxDAAQAAoFmob/eeNZtl2LI5hkTABwBAc0MABwAAgBbJ\nUcpn6wv46iunlQjsAABwdARwAAAAwG2wVXMMyTLgq6ucVqp9/h276wAAcDwEcAAAAIADqqtb7a26\n1zpC+SzltgAA1EYABwAAANzlHKl81pbdawnvAAB3KwI4AAAAoIWxaflsI8aN7V7raLv5CAABAA1F\nAAcAAADAbhrVvbbm/XbezVdfAEhABwCoRgAHAAAA4K5l1918dQSAlNcCAG5EAAcAAAAAt6m5ltfe\nOK4ZFkoEhADQWARwAAAAAGADd3N57Y3jmuFhfbv72M0HALU5RABXVlamuLg47d69Wy4uLpo1a5Yi\nIiJuem92drbi4uJ06tQp9e7dW3Fxcerfv78x/8knnyg5OVmFhYUKCQnR0qVL1alTJ2M+KSlJ27dv\nV2VlpX7xi19o/vz5Nl8fAAAAADSWIzXLqBkW3mp3n6Pv5qs5rq80mHARgLU4RAC3cuVKZWdna+vW\nrbpw4YIWLFggHx8fjRo1yuK+kpISPf/88xo/frwSExOVlpamyMhI7dmzRy4uLsrKytKrr76qJUuW\nyM/PT0uXLtWiRYv0zjvvSJJ+85vf6LPPPtP69etlNpsVHR0tT0/PW4Z9AAAAAIDa6tvd56i7+erb\n3efopcIEfsDdy+4BXElJiT788ENt3rxZfn5+8vPz0+zZs/Xuu+/WCuA+/fRT3XPPPcautcWLF+vv\nf/+7MjIyFB4ervfee09jxozRuHHjJElvvPGGQkNDlZubKx8fH23dulXz5s1TUFCQJCk6OlrJyckE\ncAAAAABgQ46ym6/muCGlwY4ULjYm8Gvq7j7CQ8C67B7A5eTkqKKiQoGBgca1wYMHa8OGDbXuzcrK\n0uDBgy2uDRo0SIcPH1Z4eLiOHDmiyMhIY65r167y9vbW0aNH1aZNG+Xn5ys4ONjic/Ly8lRUVCRP\nT08brA4AAAAA0Jw5Sudda+/us2d4WN/Y0cJFwkk0hN0DuMLCQnXs2FEm0w+v4uHhodLSUv3nP/+x\nOL+toKBA9913n8Xve3h46Ouvvzae5eXlZTHv6empixcvqrCwUK1atbKY9/T0VFVVlS5evEgABwAA\nAABweHdsd5+1x1bcLeho4aI1w8nmFC7SQdmS3QO4kpKSWl9+9bisrMzi+rVr1256b/V9dc2XlJRY\nPLuuzwEAAAAAANblMKXDjja+IZxsTuFiUzso23ts7bCwVVVVVZXVnnYbMjIytGzZMu3du9e4dubM\nGT3++OPav3+/OnToYFyPjIxUnz599Otf/9q4lpSUpLNnz2r9+vUKCgrS2rVrFRISYsxPmDBBYWFh\nGjx4sCZMmKCjR48aX2BpaakCAgKUnp6uvn371vuu/v7+Ki8vl7u7u8y5uWojySxJXl5SQYFVxlcl\ntfn/H9UztvZn23rcUtdm73fj74m1sTbHGTentbA21uZo4+a0FtbG2hxt3JzWwtpYm6OPm9Pabvbu\nVewpGscAAAwcSURBVPphJ1ippNb/H5dLauvlpbKCAocZu/r46EbOzs4qLS21GOfn58tkMikrK0v1\nsXsAd/jwYU2bNk1ZWVlycnKSJO3fv18vvPCCDh8+bHFvTEyMzGazEhISjGsLFy6Us7Oz4uPj9dhj\nj2nOnDkKDw835keMGKHo6GgFBwdr+PDh+vOf/6xu3bpJki5cuKCRI0fqH//4R4NKUIODg1VaWqqa\nZa4AAAAAAABoWQoKCuTs7KzMzMx677V7CWrfvn1lMpl05MgRDRo0SJKUmZmpAQMG1Lo3ICBAKSkp\nFtcOHTqkX/7yl5KkwMBAHTx40Ajg8vPzdfHiRQUGBsrLy0vdunXTwYMHjQAuMzNT3t7eDT7/rSFf\nKAAAAAAAAHCj1nFxcXH2fAGTyaT8/HylpaVp4MCBOnbsmJKSkhQdHa2ePXuqqKhIrVu3lslkUvfu\n3bV582ZdunRJ3bp10/r165WTk6MlS5bIZDKpc+fOSkxMVOfOneXk5KTY2Fj16dNHkyZNknS95HTD\nhg3q37+/Lly4oCVLligiIsKiAysAAAAAAABgTXYvQZWuN0+Ij4/X559/Ljc3N82ePVvTpk2TJPn5\n+SkxMdHY1Xbs2DHFxsbq7Nmz6tOnj+Lj4+Xn52c8a+fOnUpOTtaVK1cUEhKipUuX6t5775UkVVZW\n6o033tCOHTvUunVrPf3004qKirrzCwYAAAAAAECL4RABHAAAAAAAANBcOdn7BQAAAAAAAIDmjAAO\nAAAAAAAAsCECOAAAABuZNm2a/Pz8LP4MHDhQoaGhWrp0qUpLS636eSNGjNBbb71l1WdOmzZNixYt\nuuX88ePHNWbMGPn7++v111+3ymeWlJTovffes8qzAAAAHIHJ3i8AAADQnIWFhenVV19V9bG7V69e\n1d69e7V8+XJVVVUpJibGzm/YNBs2bJCzs7M+++wzubm5WeWZmzdvVnp6uqZOnWqV5wEAANgbO+AA\nAABsyNnZWe7u7vLw8JCHh4d8fX01efJkjRs3Tp9++qm9X6/Jrly5or59++pHP/qR0Xm+qegRBgAA\nmhsCOAAAADto27at2rRpY4zz8/MVFRWlBx98UAMGDNDDDz+spKQkYz49PV2jRo0y/jtw4EA9+eST\nOnTo0E2ff/XqVU2ePFnh4eG6fPmyJOnMmTN6/vnnFRQUpJCQEEVHR6uoqMj4nbKyMq1YsUIPPvig\n7r//fiUlJdUZho0YMUIHDhxQenq6+vbtq7y8PElSSkqKHn30UQUGBuqJJ57Qrl27LH5vz549mjBh\ngoKCguTv768nn3xSe/fulSS99dZbWrdunXJzc41nLly4UNOnT7d4xqJFi4xrubm58vPz08aNGxUS\nEqKRI0fq+++/V3FxsV577TUNGzZMwcHBmjlzpo4fP24849q1a1q8eLFCQkLk7++vJ554Qrt37771\nXxoAAMBtIoADAAC4gyoqKvTXv/5VH3/8scaPH29cnzNnjr7//nv99re/VUZGhp599llt2rRJf/rT\nn4x78vLy9P777yspKUk7d+5Uu3btbno+27Vr1xQZGamysjJt3bpVHTt2VEFBgaZOnaoePXooPT1d\nGzduVHFxsSZOnKhr165JkpYtW6aMjAytXLlSv//975Wfn6/MzMxbrmX79u0KCAhQWFiY9u3bp65d\nu2rVqlV6//33FRMTo127dmn69OmKj49XWlqaJOmrr77Siy++qLFjx+qTTz7Rtm3b5OHhoQULFqi8\nvFzPPvusIiIi5O3tbTyzVatWDfpud+7cqdTUVK1evVqurq6aPXu28vLytHHjRn3wwQcKCAjQ5MmT\nlZOTI0lavXq1Tp8+rU2bNukPf/iDhg8frqioKCNIBAAAsBbOgAMAALChXbt2KSMjwxiXlpbKx8dH\nzz33nCIjI41r4eHhGjNmjLp06SJJmj59ujZu3KhTp07p5z//uaTr4V18fLz69OkjSYqIiNDcuXNV\nVFQkT09P41kvvPCCSkpKlJqaqvbt20uS0tLS5O3tbRHYvfnmmxo2bJgyMjI0cuRIpaenKz4+Xg89\n9JAkKSEhQfv377/l2jp16qQ2bdoYZbbVn7lq1SoNHz5ckuTr66sLFy4oJSVFkydPVuvWrRUTE6NJ\nkyYZz5k2bZoiIyP17bffqkuXLnJ1dZWTk5Pc3d0b9V1PnTpVvXr1kiT961//UlZWlr744gt16NBB\nkhQVFaVDhw4pNTVVCQkJOn/+vFxdXeXj4yM3NzfNmzdPQ4YMMe4HAACwFgI4AAAAGxoxYoTmz5+v\nqqoqZWVlafny5Ro2bJgiIyPl5HS9GMHZ2VlTpkzR559/rqNHj+rcuXM6efKkvv32W1VUVFg8r2fP\nnsbP1U0PzGazcS01NVXl5eUaOnSoEb5JUnZ2tk6fPq2goCCL55WVlenMmTP66U9/qvLycg0YMMCY\na9u2rfr169fgtX799dcqLS1VdHS0xfXKykqZzWaVlZXJz89P9957r1JSUnT27Fl98803OnHihCTV\nWmtjde/e3fg5OztblZWVevjhhy3uMZvNxvf13HPPac6cORo2bJj8/f31s5/9TGPHjrX43gAAAKyB\nAA4AAMCGXF1d5evrK+l6QNS5c2dFRETIZDIZHVBLSko0depUlZWVafTo0QoODpa/v7+mTJlS63k3\nnhtX7cZz2vz8/PTyyy9r5syZ+uCDD/T0009Luh6CPfDAA4qLi6v1+25ubsrNzVVVVZUqKyst5kym\nhv9zsfo9Vq9ebREUVmvbtq2+/PJLzZ49W4888ogGDx6scePG6erVq5o7d26DP0eSysvLa11zcXEx\nfq6srJSbm5t27Nhx0/eQpMDAQP3tb3/Tvn379M9//lMfffSR3n77bW3atElDhw5t1PsAAADUhTPg\nAAAA7qAHHnhAERERSktLMxoP7N27VydOnNDvfvc7zZ07V6NHj1a7du0sGiQ01PDhwxUcHKyIiAit\nXLlSly5dkiT17t1bZ8+eVdeuXeXr6ytfX1916NBBy5cv16lTp9SjRw85OztbNHWoqKgwzktriJ49\ne8pkMikvL8/4DF9fX/3lL3/R5s2bJUlbtmzR0KFDtWbNGs2YMUPDhg0zzly7VcOHNm3aqLi42OLa\nN998U+e73HfffSouLlZZWZnFu2zYsEF79uyRJK1du1aZmZkKDQ3V4sWLlZGRIV9fX/3xj39s8JoB\nAAAaggAOAADgDps3b566d++u2NhYlZSUGOe+ffTRR8rLy1NmZqZ+9atfqaKiQmVlZXU+61ah1dy5\nc+Xu7q7FixdLkqZMmaL//ve/io6OVk5OjnJycvTSSy/p+PHj6t27t9q1a6dnnnlGa9eu1e7du3X2\n7FnFxMQYAV5DtG/fXpMmTdLq1av18ccf6/z58/rwww+VlJQkLy8vSZK3t7dOnjypgwcPKjc3V9u3\nb9eaNWskyVirq6urvvvuO/373/9WeXm5AgMDdfLkSe3atUsXLlzQunXrdOrUqTrf5aGHHpKfn5+i\noqK0f/9+nTt3TgkJCdq5c6d69+4tSTp//rzi4uL0xRdfKC8vTxkZGcrPz9egQYMavGYAAICGoAQV\nAADgDmvbtq2WLVumGTNm6M0339Qrr7yihQsXKjU1VcnJyerSpYvCwsLk7e2tY8eO1fmsGzuE3viz\ns7Oz4uPjNWvWLG3fvl1PPfWU3n33XSUlJWnKlCkymUwaNGiQUlNT1alTJ0lSdHS0XFxcFB8fr6tX\nryosLMxoANFQr7zyitzd3bVmzRoVFBTI29tbL730kmbNmiVJevHFF1VUVKQ5c+ZIknr16qWEhATN\nnz9fx44dU48ePTRq1Cht27ZN48eP19atWzVu3DidOHFCy5YtU0VFhcaMGaMZM2bo8OHDN127JDk5\nOWnLli16/fXXFRUVpZKSEvXq1Uvr1q3TkCFDJEmxsbFauXKlXn75ZV2+fFk+Pj6aP3++Hn/88Uat\nGQAAoD6tqm71v00BAAAAAAAANBklqAAAAAAAAIANEcABAAAAAAAANkQABwAAAAAAANgQARwAAAAA\nAABgQwRwAAAAAAAAgA0RwAEAAAAAAAA2RAAHAAAAAAAA2BABHAAAAAAAAGBDBHAAAAAAAACADRHA\nAQAAAAAAADZEAAcAAAAAAADYEAEcAAAAAAAAYEP/AwUew+yLy8DFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2d8e6f385f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.set_style('white')\n",
    "fig,ax = plt.subplots(figsize=(15,5))\n",
    "ax.bar(range(X.shape[1]), importances[indices],color=\"r\", align=\"center\")\n",
    "plt.ylabel(\"Feature importance\")\n",
    "plt.xlabel('Ranked features')\n",
    "plt.xticks([], indices)\n",
    "plt.xlim([-1, X.shape[1]]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Features derived from raw geological variables tend to have the highest classification power. Rolling min, max and mean tend to have better classification power than raw data. Wavelet approximation coeficients tend to have a similar to lower classification power than raw data. Features expressing local texture of the data (entropy, gradient, standard deviation and wavelet detail coeficients) have a low classification power but still participate in the prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Confusion matrix\n",
    "The confusion matrix from the validation test is presented below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SS</th>\n",
       "      <th>CSiS</th>\n",
       "      <th>FSiS</th>\n",
       "      <th>SiSh</th>\n",
       "      <th>MS</th>\n",
       "      <th>WS</th>\n",
       "      <th>D</th>\n",
       "      <th>PS</th>\n",
       "      <th>BS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>SS</th>\n",
       "      <td>170</td>\n",
       "      <td>78</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CSiS</th>\n",
       "      <td>77</td>\n",
       "      <td>604</td>\n",
       "      <td>254</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>FSiS</th>\n",
       "      <td>25</td>\n",
       "      <td>255</td>\n",
       "      <td>484</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SiSh</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>177</td>\n",
       "      <td>25</td>\n",
       "      <td>38</td>\n",
       "      <td>18</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MS</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>9</td>\n",
       "      <td>53</td>\n",
       "      <td>30</td>\n",
       "      <td>105</td>\n",
       "      <td>26</td>\n",
       "      <td>50</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WS</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>71</td>\n",
       "      <td>86</td>\n",
       "      <td>225</td>\n",
       "      <td>35</td>\n",
       "      <td>150</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>D</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>16</td>\n",
       "      <td>3</td>\n",
       "      <td>9</td>\n",
       "      <td>77</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PS</th>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>11</td>\n",
       "      <td>35</td>\n",
       "      <td>15</td>\n",
       "      <td>159</td>\n",
       "      <td>73</td>\n",
       "      <td>341</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BS</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>15</td>\n",
       "      <td>54</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       SS  CSiS  FSiS  SiSh  MS   WS   D   PS   BS\n",
       "SS    170    78    20     0   0    0   0    0    0\n",
       "CSiS   77   604   254     3   0    2   0    0    0\n",
       "FSiS   25   255   484     8   0    2   1    5    0\n",
       "SiSh    0     2     3   177  25   38  18    8    0\n",
       "MS      0     3     9    53  30  105  26   50   20\n",
       "WS      0     0     4    71  86  225  35  150   11\n",
       "D       0     0     1    16   3    9  77   35    0\n",
       "PS      0     4    11    35  15  159  73  341   48\n",
       "BS      0     2     0     0   1   13  15   54  100"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "######## Confusion matrix from this tuning\n",
    "cv=LeaveOneGroupOut().split(X, y, groups)\n",
    "y_pred = cross_val_predict(Cl, X, y, cv=cv, n_jobs=-1)\n",
    "conf_mat = confusion_matrix(y, y_pred)\n",
    "list_facies = ['SS', 'CSiS', 'FSiS', 'SiSh', 'MS', 'WS', 'D', 'PS', 'BS']\n",
    "conf_mat = pd.DataFrame(conf_mat, columns=list_facies, index=list_facies)\n",
    "conf_mat.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applying the classification model to test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Formation</th>\n",
       "      <th>Well Name</th>\n",
       "      <th>Depth</th>\n",
       "      <th>GR</th>\n",
       "      <th>ILD_log10</th>\n",
       "      <th>DeltaPHI</th>\n",
       "      <th>PHIND</th>\n",
       "      <th>PE</th>\n",
       "      <th>NM_M</th>\n",
       "      <th>RELPOS</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2808.0</td>\n",
       "      <td>66.276</td>\n",
       "      <td>0.630</td>\n",
       "      <td>3.3</td>\n",
       "      <td>10.65</td>\n",
       "      <td>3.591</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2808.5</td>\n",
       "      <td>77.252</td>\n",
       "      <td>0.585</td>\n",
       "      <td>6.5</td>\n",
       "      <td>11.95</td>\n",
       "      <td>3.341</td>\n",
       "      <td>1</td>\n",
       "      <td>0.978</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2809.0</td>\n",
       "      <td>82.899</td>\n",
       "      <td>0.566</td>\n",
       "      <td>9.4</td>\n",
       "      <td>13.60</td>\n",
       "      <td>3.064</td>\n",
       "      <td>1</td>\n",
       "      <td>0.956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2809.5</td>\n",
       "      <td>80.671</td>\n",
       "      <td>0.593</td>\n",
       "      <td>9.5</td>\n",
       "      <td>13.25</td>\n",
       "      <td>2.977</td>\n",
       "      <td>1</td>\n",
       "      <td>0.933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2810.0</td>\n",
       "      <td>75.971</td>\n",
       "      <td>0.638</td>\n",
       "      <td>8.7</td>\n",
       "      <td>12.35</td>\n",
       "      <td>3.020</td>\n",
       "      <td>1</td>\n",
       "      <td>0.911</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Formation Well Name   Depth      GR  ILD_log10  DeltaPHI  PHIND     PE  \\\n",
       "0     A1 SH    STUART  2808.0  66.276      0.630       3.3  10.65  3.591   \n",
       "1     A1 SH    STUART  2808.5  77.252      0.585       6.5  11.95  3.341   \n",
       "2     A1 SH    STUART  2809.0  82.899      0.566       9.4  13.60  3.064   \n",
       "3     A1 SH    STUART  2809.5  80.671      0.593       9.5  13.25  2.977   \n",
       "4     A1 SH    STUART  2810.0  75.971      0.638       8.7  12.35  3.020   \n",
       "\n",
       "   NM_M  RELPOS  \n",
       "0     1   1.000  \n",
       "1     1   0.978  \n",
       "2     1   0.956  \n",
       "3     1   0.933  \n",
       "4     1   0.911  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filename = '../validation_data_nofacies.csv'\n",
    "test_data = pd.read_csv(filename)\n",
    "test_data.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "##### cD From wavelet db1\n",
    "dwt_db1_cD_df = FE.make_dwt_vars_cD(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                    levels=[1, 2, 3, 4], wavelet='db1')\n",
    "\n",
    "##### cA From wavelet db1\n",
    "dwt_db1_cA_df = FE.make_dwt_vars_cA(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                       levels=[1, 2, 3, 4], wavelet='db1')\n",
    "\n",
    "##### cD From wavelet db3\n",
    "dwt_db3_cD_df = FE.make_dwt_vars_cD(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                       levels=[1, 2, 3, 4], wavelet='db3')\n",
    "\n",
    "##### cA From wavelet db3\n",
    "dwt_db3_cA_df = FE.make_dwt_vars_cA(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                       levels=[1, 2, 3, 4], wavelet='db3')\n",
    "\n",
    "##### From entropy\n",
    "entropy_df = FE.make_entropy_vars(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                               l_foots=[2, 3, 4, 5, 7, 10])\n",
    "\n",
    "###### From gradient\n",
    "gradient_df = FE.make_gradient_vars(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                 dx_list=[2, 3, 4, 5, 6, 10, 20])\n",
    "\n",
    "##### From rolling average\n",
    "moving_av_df = FE.make_moving_av_vars(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                   windows=[1, 2, 5, 10, 20])\n",
    "\n",
    "##### From rolling standard deviation\n",
    "moving_std_df = FE.make_moving_std_vars(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                     windows=[3 , 4, 5, 7, 10, 15, 20])\n",
    "\n",
    "##### From rolling max\n",
    "moving_max_df = FE.make_moving_max_vars(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                     windows=[3, 4, 5, 7, 10, 15, 20])\n",
    "\n",
    "##### From rolling min\n",
    "moving_min_df = FE.make_moving_min_vars(wells_df=test_data, logs=['GR', 'ILD_log10', 'DeltaPHI', 'PE', 'PHIND'],\n",
    "                                     windows=[3 , 4, 5, 7, 10, 15, 20])\n",
    "\n",
    "###### From rolling NM/M ratio\n",
    "rolling_marine_ratio_df = FE.make_rolling_marine_ratio_vars(wells_df=test_data, windows=[5, 10, 15, 20, 30, 50, 75, 100, 200])\n",
    "\n",
    "###### From distance to NM and M, up and down\n",
    "dist_M_up_df = FE.make_distance_to_M_up_vars(wells_df=test_data)\n",
    "dist_M_down_df = FE.make_distance_to_M_down_vars(wells_df=test_data)\n",
    "dist_NM_up_df = FE.make_distance_to_NM_up_vars(wells_df=test_data)\n",
    "dist_NM_down_df = FE.make_distance_to_NM_down_vars(wells_df=test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(830, 298)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Formation</th>\n",
       "      <th>Well Name</th>\n",
       "      <th>Depth</th>\n",
       "      <th>GR</th>\n",
       "      <th>ILD_log10</th>\n",
       "      <th>DeltaPHI</th>\n",
       "      <th>PHIND</th>\n",
       "      <th>PE</th>\n",
       "      <th>NM_M</th>\n",
       "      <th>RELPOS</th>\n",
       "      <th>...</th>\n",
       "      <th>Marine_ratio_20_centered</th>\n",
       "      <th>Marine_ratio_30_centered</th>\n",
       "      <th>Marine_ratio_50_centered</th>\n",
       "      <th>Marine_ratio_75_centered</th>\n",
       "      <th>Marine_ratio_100_centered</th>\n",
       "      <th>Marine_ratio_200_centered</th>\n",
       "      <th>dist_M_up</th>\n",
       "      <th>dist_M_down</th>\n",
       "      <th>dist_NM_up</th>\n",
       "      <th>dist_NM_down</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2808.0</td>\n",
       "      <td>66.276</td>\n",
       "      <td>0.630</td>\n",
       "      <td>3.3</td>\n",
       "      <td>10.65</td>\n",
       "      <td>3.591</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.140000</td>\n",
       "      <td>1.570000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>21.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2808.5</td>\n",
       "      <td>77.252</td>\n",
       "      <td>0.585</td>\n",
       "      <td>6.5</td>\n",
       "      <td>11.95</td>\n",
       "      <td>3.341</td>\n",
       "      <td>1</td>\n",
       "      <td>0.978</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.156863</td>\n",
       "      <td>1.574257</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2809.0</td>\n",
       "      <td>82.899</td>\n",
       "      <td>0.566</td>\n",
       "      <td>9.4</td>\n",
       "      <td>13.60</td>\n",
       "      <td>3.064</td>\n",
       "      <td>1</td>\n",
       "      <td>0.956</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.173077</td>\n",
       "      <td>1.578431</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>20.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2809.5</td>\n",
       "      <td>80.671</td>\n",
       "      <td>0.593</td>\n",
       "      <td>9.5</td>\n",
       "      <td>13.25</td>\n",
       "      <td>2.977</td>\n",
       "      <td>1</td>\n",
       "      <td>0.933</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.188679</td>\n",
       "      <td>1.582524</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A1 SH</td>\n",
       "      <td>STUART</td>\n",
       "      <td>2810.0</td>\n",
       "      <td>75.971</td>\n",
       "      <td>0.638</td>\n",
       "      <td>8.7</td>\n",
       "      <td>12.35</td>\n",
       "      <td>3.020</td>\n",
       "      <td>1</td>\n",
       "      <td>0.911</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.203704</td>\n",
       "      <td>1.586538</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>19.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 298 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  Formation Well Name   Depth      GR  ILD_log10  DeltaPHI  PHIND     PE  \\\n",
       "0     A1 SH    STUART  2808.0  66.276      0.630       3.3  10.65  3.591   \n",
       "1     A1 SH    STUART  2808.5  77.252      0.585       6.5  11.95  3.341   \n",
       "2     A1 SH    STUART  2809.0  82.899      0.566       9.4  13.60  3.064   \n",
       "3     A1 SH    STUART  2809.5  80.671      0.593       9.5  13.25  2.977   \n",
       "4     A1 SH    STUART  2810.0  75.971      0.638       8.7  12.35  3.020   \n",
       "\n",
       "   NM_M  RELPOS      ...       Marine_ratio_20_centered  \\\n",
       "0     1   1.000      ...                            1.0   \n",
       "1     1   0.978      ...                            1.0   \n",
       "2     1   0.956      ...                            1.0   \n",
       "3     1   0.933      ...                            1.0   \n",
       "4     1   0.911      ...                            1.0   \n",
       "\n",
       "   Marine_ratio_30_centered  Marine_ratio_50_centered  \\\n",
       "0                       1.0                       1.0   \n",
       "1                       1.0                       1.0   \n",
       "2                       1.0                       1.0   \n",
       "3                       1.0                       1.0   \n",
       "4                       1.0                       1.0   \n",
       "\n",
       "   Marine_ratio_75_centered  Marine_ratio_100_centered  \\\n",
       "0                       1.0                   1.140000   \n",
       "1                       1.0                   1.156863   \n",
       "2                       1.0                   1.173077   \n",
       "3                       1.0                   1.188679   \n",
       "4                       1.0                   1.203704   \n",
       "\n",
       "   Marine_ratio_200_centered  dist_M_up  dist_M_down  dist_NM_up  dist_NM_down  \n",
       "0                   1.570000       -1.0         21.5         0.0           0.0  \n",
       "1                   1.574257       -1.0         21.0         0.0           0.0  \n",
       "2                   1.578431       -1.0         20.5         0.0           0.0  \n",
       "3                   1.582524       -1.0         20.0         0.0           0.0  \n",
       "4                   1.586538       -1.0         19.5         0.0           0.0  \n",
       "\n",
       "[5 rows x 298 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combined_test_df = test_data\n",
    "list_df_var = [dwt_db1_cD_df, dwt_db1_cA_df, dwt_db3_cD_df, dwt_db3_cA_df,\n",
    "               entropy_df, gradient_df, moving_av_df, moving_std_df, moving_max_df, moving_min_df,\n",
    "              rolling_marine_ratio_df, dist_M_up_df, dist_M_down_df, dist_NM_up_df, dist_NM_down_df]\n",
    "for var_df in list_df_var:\n",
    "    temp_df = var_df\n",
    "    combined_test_df = pd.concat([combined_test_df,temp_df],axis=1)\n",
    "combined_test_df.replace(to_replace=np.nan, value='-99999', inplace=True)\n",
    "\n",
    "X_test = combined_test_df.iloc[:, 3:]\n",
    "\n",
    "print (combined_test_df.shape)\n",
    "combined_test_df.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Well Name</th>\n",
       "      <th>Depth</th>\n",
       "      <th>Predicted Facies</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>STUART</td>\n",
       "      <td>2808.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>STUART</td>\n",
       "      <td>2808.5</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>STUART</td>\n",
       "      <td>2809.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>STUART</td>\n",
       "      <td>2809.5</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>STUART</td>\n",
       "      <td>2810.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Well Name   Depth  Predicted Facies\n",
       "0    STUART  2808.0                 3\n",
       "1    STUART  2808.5                 3\n",
       "2    STUART  2809.0                 3\n",
       "3    STUART  2809.5                 3\n",
       "4    STUART  2810.0                 3"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Cl = RandomForestClassifier(n_estimators=100, max_features=0.1, min_samples_leaf=25,\n",
    "                            min_samples_split=50, class_weight='balanced', random_state=42, n_jobs=-1)\n",
    "Cl.fit(X, y)\n",
    "y_test = Cl.predict(X_test)\n",
    "y_test = pd.DataFrame(y_test, columns=['Predicted Facies'])\n",
    "test_pred_df = pd.concat([combined_test_df[['Well Name', 'Depth']], y_test], axis=1)\n",
    "test_pred_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exporting results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test_pred_df.to_pickle('Prediction_blind_wells_RF_c.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}