Skip to content

Commit

Permalink
fix: Installation of HDFS on Ubuntu (#11420)
Browse files Browse the repository at this point in the history
Summary:
The setup-adapters.sh uses yum when installing OpenJDK 8 during the HDFS installation. This causes errors on non-RHEL/Centos based Linux distributions such as Ubuntu.

The fix is to check for the distribution and select the corresponding package name and package manager.

Addresses: prestodb/presto#23939

Pull Request resolved: #11420

Reviewed By: Yuhta

Differential Revision: D65612082

Pulled By: kevinwilfong

fbshipit-source-id: 1bacde35c4daa87291b198596f61992f290e080f
  • Loading branch information
czentgr authored and facebook-github-bot committed Nov 7, 2024
1 parent 8c69b8b commit 327fa89
Showing 1 changed file with 7 additions and 4 deletions.
11 changes: 7 additions & 4 deletions scripts/setup-adapters.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ source $SCRIPTDIR/setup-helper-functions.sh
DEPENDENCY_DIR=${DEPENDENCY_DIR:-$(pwd)/deps-download}
CMAKE_BUILD_TYPE="${BUILD_TYPE:-Release}"
MACHINE=$(uname -m)
LINUX_DISTRIBUTION=$(. /etc/os-release && echo ${ID})

if [[ "$OSTYPE" == darwin* ]]; then
export INSTALL_PREFIX=${INSTALL_PREFIX:-"$(pwd)/deps-install"}
Expand Down Expand Up @@ -124,7 +125,7 @@ function install_azure-storage-sdk-cpp {
sed -i "s/\"version-string\"/\"overrides\": [{ \"name\": \"openssl\", \"version-string\": \"$openssl_version\" }],\"version-string\"/" $azure_core_dir/vcpkg.json
fi
(
cd $azure_core_dir
cd $azure_core_dir
cmake_install -DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE} -DBUILD_SHARED_LIBS=OFF
)
# install azure-storage-common
Expand Down Expand Up @@ -161,8 +162,11 @@ function install_hdfs_deps {
cp -a ${DEPENDENCY_DIR}/hadoop /usr/local/
wget -P /usr/local/hadoop/share/hadoop/common/lib/ https://repo1.maven.org/maven2/junit/junit/4.11/junit-4.11.jar

yum install -y java-1.8.0-openjdk-devel

if [[ "$LINUX_DISTRIBUTION" == "ubuntu" || "$LINUX_DISTRIBUTION" == "debian" ]]; then
apt install -y openjdk-8-jdk
else # Assume Fedora/CentOS
yum install -y java-1.8.0-openjdk-devel
fi
fi
cmake_install_dir $libhdfs3_dir
}
Expand All @@ -173,7 +177,6 @@ function install_hdfs_deps {
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
# /etc/os-release is a standard way to query various distribution
# information and is available everywhere
LINUX_DISTRIBUTION=$(. /etc/os-release && echo ${ID})
if [[ "$LINUX_DISTRIBUTION" == "ubuntu" || "$LINUX_DISTRIBUTION" == "debian" ]]; then
apt install -y --no-install-recommends libxml2-dev libgsasl7-dev uuid-dev
# Dependencies of GCS, probably a workaround until the docker image is rebuilt
Expand Down

0 comments on commit 327fa89

Please sign in to comment.