From a79b41c71cc845ad67703be5f1f1646d7c737e23 Mon Sep 17 00:00:00 2001 From: Saurabh Ahuja Date: Tue, 20 Feb 2024 23:12:17 +0530 Subject: [PATCH] fix-nfs (#2759) * fix-nfs * permissive policy update * add cleanup scripts * add cleanup scripts --- OracleDatabase/RAC/OracleDNSServer/README.md | 8 +-- .../RAC/OracleRACStorageServer/README.md | 6 ++ .../OracleRealApplicationClusters/README.md | 2 +- .../samples/racdockercompose/README.md | 33 +++++++++++ .../samples/racpodmancompose/README.md | 58 ++++++++++++++++--- 5 files changed, 93 insertions(+), 14 deletions(-) diff --git a/OracleDatabase/RAC/OracleDNSServer/README.md b/OracleDatabase/RAC/OracleDNSServer/README.md index 200102dac7..d77afc0847 100644 --- a/OracleDatabase/RAC/OracleDNSServer/README.md +++ b/OracleDatabase/RAC/OracleDNSServer/README.md @@ -38,8 +38,8 @@ docker network create --driver=bridge --subnet=192.168.17.0/24 rac_priv1_nw ``` **Note:** You can change the subnet according to your environment. -### Running RAC DNS server container -Execute following command to create the container on Docker Host: +## Running RAC DNS server container +### Execute following command to create the container on Docker Host ```bash docker create --hostname racdns \ @@ -61,7 +61,7 @@ docker network connect rac_priv1_nw --ip 192.168.17.25 rac-dnsserver docker start rac-dnsserver ``` -Execute following command to create the container on Podman Host: +### Execute following command to create the container on Podman Host ```bash podman create --hostname racdns \ @@ -99,4 +99,4 @@ you should see the following in docker logs output: ################################################# DNS Server IS READY TO USE! ################################################# -``` +``` \ No newline at end of file diff --git a/OracleDatabase/RAC/OracleRACStorageServer/README.md b/OracleDatabase/RAC/OracleRACStorageServer/README.md index df048c8b42..77c2cde84f 100644 --- a/OracleDatabase/RAC/OracleRACStorageServer/README.md +++ b/OracleDatabase/RAC/OracleRACStorageServer/README.md @@ -136,6 +136,12 @@ mkdir -p /scratch/stage/rac-storage/$ORACLE_DBNAME rm -rf /scratch/stage/rac-storage/$ORACLE_DBNAME/asm_disk0* ``` +If SELinux is enabled on Podman Host (you can check by running `sestatus` command), then execute below to make SELinux policy as `permissive` and reboot the host machine. This will allow permissions to write to `asm-disks*` in the `/oradata` folder inside the podman containers- +```bash +sed -i 's/^SELINUX=.*/SELINUX=permissive/' /etc/selinux/config +reboot +``` + Execute following command to create the container: ```bash diff --git a/OracleDatabase/RAC/OracleRealApplicationClusters/README.md b/OracleDatabase/RAC/OracleRealApplicationClusters/README.md index 3828ce4867..8f3bd66ed0 100644 --- a/OracleDatabase/RAC/OracleRealApplicationClusters/README.md +++ b/OracleDatabase/RAC/OracleRealApplicationClusters/README.md @@ -623,7 +623,7 @@ You must install and configure [Podman release 4.0.2](https://docs.oracle.com/en - You can check the details on [Oracle Linux and Unbreakable Enterprise Kernel (UEK) Releases](https://blogs.oracle.com/scoter/post/oracle-linux-and-unbreakable-enterprise-kernel-uek-releases) -- You do not need to execute step 2 in this section to create and enable `Podman-rac-cgroup.service` when we are running Oracle Linux 8 with Unbreakable Enterprise Kernel R7. +- You do not need to execute step 2 in this section to create and enable `podman-rac-cgroup.service` when we are running Oracle Linux 8 with Unbreakable Enterprise Kernel R7. **IMPORTANT:** Completing prerequisite steps is a requirement for successful configuration. diff --git a/OracleDatabase/RAC/OracleRealApplicationClusters/samples/racdockercompose/README.md b/OracleDatabase/RAC/OracleRealApplicationClusters/samples/racdockercompose/README.md index e9b759185d..f881df0ad3 100644 --- a/OracleDatabase/RAC/OracleRealApplicationClusters/samples/racdockercompose/README.md +++ b/OracleDatabase/RAC/OracleRealApplicationClusters/samples/racdockercompose/README.md @@ -14,6 +14,7 @@ Once you have built your Oracle RAC container image, you can create a Oracle RAC - [Section 4.1: Add Additional Node in Existing Oracle RAC Cluster with Block Devices](#section-41-add-additional-node-in-existing-oracle-rac-cluster-with-block-devices) - [Section 4.2: Add Additional Node in Existing Oracle RAC Cluster with NFS Volume](#section-42-add-additional-node-in-existing-oracle-rac-cluster-with-nfs-volume) - [Section 5: Connect to the RAC container](#connect-to-the-rac-container) + - [Cleanup RAC Environment](#cleanup-rac-environment) - [Copyright](#copyright) ## Section 1 : Prerequisites for RAC Database on Docker with Docker Compose @@ -451,6 +452,38 @@ docker exec -i -t racnoded1 /bin/bash If the install fails for any reason, log in to container using the above command and check `/tmp/orod.log`. You can also review the Grid Infrastructure logs located at `$GRID_BASE/diag/crs` and check for failure logs. If the failure occurred during the database creation then check the database logs. +## Cleanup RAC Environment +Below commands can be executed to cleanup above RAC Environment - + +### Cleanup RAC based on Block Devices +```bash +#----Cleanup RAC Containers----- +docker rm -f racnoded1 racnoded2 rac-dnsserver racnodedc1-cman +#----Cleanup Disks-------------- +dd if=/dev/zero of=/dev/oracleoci/oraclevde bs=8k count=10000 status=progress && dd if=/dev/zero of=/dev/oracleoci/oraclevdd bs=8k count=10000 status=progress +#----Cleanup Files and Folders-- +rm -rf /opt/containers /opt/.secrets +#----Cleanup Docker Networks-- +docker network rm -f rac_pub1_nw rac_pzriv1_nw +#----Cleanup Docker Images-- +docker rmi -f oracle/rac-dnsserver:latest oracle/database-rac:19.3.0 oracle/client-cman:19.3.0 +``` + +### Cleanup RAC based on NFS Storage Devices +```bash +#----Cleanup RAC Containers----- +docker rm -f racnoded1 racnoded2 rac-dnsserver racnode-storage racnodedc1-cman +#----Cleanup Files and Folders-- +rm -rf /opt/containers /opt/.secrets +export ORACLE_DBNAME=ORCLCDB +rm -rf /docker_volumes/asm_vol/$ORACLE_DBNAME/asm_disk0* +#----Cleanup Docker Volumes--- +docker volume -f racstorage +#----Cleanup Docker Networks-- +docker network rm -f rac_pub1_nw rac_pzriv1_nw +#----Cleanup Docker Images-- +docker rmi -f oracle/rac-dnsserver:latest oracle/rac-storage-server:19.3.0 oracle/database-rac:19.3.0 oracle/client-cman:19.3.0 +``` ## Copyright diff --git a/OracleDatabase/RAC/OracleRealApplicationClusters/samples/racpodmancompose/README.md b/OracleDatabase/RAC/OracleRealApplicationClusters/samples/racpodmancompose/README.md index 819caeee28..bff527f177 100644 --- a/OracleDatabase/RAC/OracleRealApplicationClusters/samples/racpodmancompose/README.md +++ b/OracleDatabase/RAC/OracleRealApplicationClusters/samples/racpodmancompose/README.md @@ -14,6 +14,7 @@ Once you have built your Oracle RAC container image, you can create a Oracle RAC - [Section 4.1: Add Additional Node in Existing Oracle RAC Cluster with Block Devices](#section-41-add-additional-node-in-existing-oracle-rac-cluster-with-block-devices) - [Section 4.2: Add Additional Node in Existing Oracle RAC Cluster with NFS Volume](#section-42-add-additional-node-in-existing-oracle-rac-cluster-with-nfs-volume) - [Section 5: Connect to the RAC container](#connect-to-the-rac-container) + - [Cleanup RAC Environment](#cleanup-rac-environment) - [Copyright](#copyright) ## Section 1 : Prerequisites for RAC Database on Podman with Podman Compose @@ -92,6 +93,12 @@ In order to setup Oracle RAC on Podman with Oracle RAC Storage Container with Po yum -y install nfs-utils ``` +If SELinux is enabled on Podman Host (you can check by running `sestatus` command), then execute below to make SELinux policy as `permissive` and reboot host machine. This will allow permissions to write to `asm-disks*` in the `/oradata` folder inside the podman containers- +```bash +sed -i 's/^SELINUX=.*/SELINUX=permissive/' /etc/selinux/config +reboot +``` + Lets identify necessary variables to export that will be used by `podman-compose.yml` file later. Below is one example of exporting necessary variables related to docker network, DNS container, Storage Container, RAC Container and CMAN container discussed in this repo. ```bash export HEALTHCHECK_INTERVAL=30s @@ -241,15 +248,6 @@ mkdir -p /scratch/stage/rac-storage/$ORACLE_DBNAME rm -rf /scratch/stage/rac-storage/ORCLCDB/asm_disk0* ``` -```bash -#----------Create NFS volume-------------- -podman volume create --driver local \ ---opt type=nfs \ ---opt o=addr=192.168.17.80,rw,bg,hard,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,actimeo=0 \ ---opt device=192.168.17.80:/oradata \ -racstorage -``` - After copying compose file, you can bring up DNS Container, Storage Container, RAC Container and CMAN container by following below commands- ```bash #---------Bring up DNS------------ @@ -278,6 +276,16 @@ Export list for racnode-storage: ################################################# ``` +```bash +#----------Create NFS volume-------------- +podman volume create --driver local \ +--opt type=nfs \ +--opt o=addr=192.168.17.80,rw,bg,hard,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,actimeo=0 \ +--opt device=192.168.17.80:/oradata \ +racstorage +``` + + ```bash #-----Bring up racnode1---------- podman-compose --podman-run-args="-t -i --systemd=always --cpuset-cpus 0-1 --memory 16G --memory-swap 32G" up -d ${RACNODE1_CONTAINER_NAME} && \ @@ -444,6 +452,38 @@ podman exec -i -t racnodep1 /bin/bash If the install fails for any reason, log in to container using the above command and check `/tmp/orod.log`. You can also review the Grid Infrastructure logs located at `$GRID_BASE/diag/crs` and check for failure logs. If the failure occurred during the database creation then check the database logs. +## Cleanup RAC Environment +Below commands can be executed to cleanup above RAC Environment - + +### Cleanup RAC based on Block Devices +```bash +#----Cleanup RAC Containers----- +podman rm -f racnodep1 racnodep2 rac-dnsserver racnodepc1-cman +#----Cleanup Disks-------------- +dd if=/dev/zero of=/dev/oracleoci/oraclevde bs=8k count=10000 status=progress && dd if=/dev/zero of=/dev/oracleoci/oraclevdd bs=8k count=10000 status=progress +#----Cleanup Files and Folders-- +rm -rf /opt/containers /opt/.secrets +#----Cleanup Docker Networks-- +podman network rm -f rac_pub1_nw rac_zriv1_nw +#----Cleanup Docker Images-- +podman rmi -f localhost/oracle/rac-dnsserver:latest localhost/oracle/database-rac:21.3.0-21.13.0 localhost/oracle/client-cman:21.3.0 +``` + +### Cleanup RAC based on NFS Storage Devices +```bash +#----Cleanup RAC Containers----- +podman rm -f racnodep1 racnodep2 rac-dnsserver racnode-storage racnodepc1-cman +#----Cleanup Files and Folders-- +rm -rf /opt/containers /opt/.secrets +export ORACLE_DBNAME=ORCLCDB +rm -rf /scratch/stage/rac-storage/ORCLCDB/asm_disk0* +#----Cleanup Docker Volumes--- +podman volume -f racstorage +#----Cleanup Docker Networks-- +podman network rm -f rac_pub1_nw rac_priv1_nw +#----Cleanup Docker Images-- +podman rmi -f localhost/oracle/rac-dnsserver:latest localhost/oracle/rac-storage-server:latest localhost/oracle/database-rac:21.3.0-21.13.0 localhost/oracle/client-cman:21.3.0 +``` ## Copyright