node_sqlite3.node crashing on Windows arm64

[dennisameling]

I'm trying to get Signal Desktop to work on Windows arm64 (signalapp/Signal-Desktop#3745 (comment)), and it's crashing with error code 3228369023 when I try to run it. I debugged in Visual Studio (on an x64 machine, since Microsoft doesn't support debugging on arm64 yet) and it looks like it's crashing on node_sqlite3.node:

When I clone this repo and build it, it fails with the following error:

C:\repos\node-sqlcipher>npm install

> @journeyapps/[email protected] install C:\repos\node-sqlcipher
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(403): https://journeyapps-node-binary.s3.amazonaws.com/@journeyapps/sqlcipher/v5.0.0/napi-v6-win32-arm64.tar.gz
node-pre-gyp WARN Pre-built binaries not found for @journeyapps/[email protected] and [email protected] (node-v83 ABI, unknown) (falling back 
to source compile with node-gyp)
Building the projects in this solution one at a time. To enable parallel build, please add the "-m" switch.
  nothing.c
  win_delay_load_hook.cc
  nothing.vcxproj -> C:\repos\node-sqlcipher\build\Release\\nothing.lib
  unpack_sqlite_dep
  Copying C:\repos\node-sqlcipher\build\Release\obj/global_intermediate/sqlcipher-amalgamation-3031000/OpenSSL-Win64/msvcr120.dll to C
  :\repos\node-sqlcipher\build\Release\\msvcr120.dll
          1 file(s) copied.
  Copying C:\repos\node-sqlcipher\build\Release\obj/global_intermediate/sqlcipher-amalgamation-3031000/OpenSSL-Win64/libeay32.dll to C 
  :\repos\node-sqlcipher\build\Release\\libeay32.dll
          1 file(s) copied.
  sqlite3.c
C:\repos\node-sqlcipher\build\Release\obj\global_intermediate\sqlcipher-amalgamation-3031000\sqlite3.c(27233,1): error C2099: initiali
zer is not a constant [C:\repos\node-sqlcipher\build\deps\sqlite3.vcxproj]
C:\repos\node-sqlcipher\build\Release\obj\global_intermediate\sqlcipher-amalgamation-3031000\sqlite3.c(27234,1): error C2099: initiali 
zer is not a constant [C:\repos\node-sqlcipher\build\deps\sqlite3.vcxproj]
C:\repos\node-sqlcipher\build\Release\obj\global_intermediate\sqlcipher-amalgamation-3031000\sqlite3.c(27235,1): error C2099: initiali
zer is not a constant [C:\repos\node-sqlcipher\build\deps\sqlite3.vcxproj]
C:\repos\node-sqlcipher\build\Release\obj\global_intermediate\sqlcipher-amalgamation-3031000\sqlite3.c(27237,1): error C2099: initiali 
zer is not a constant [C:\repos\node-sqlcipher\build\deps\sqlite3.vcxproj]
C:\repos\node-sqlcipher\build\Release\obj\global_intermediate\sqlcipher-amalgamation-3031000\sqlite3.c(27238,1): error C2099: initiali 
zer is not a constant [C:\repos\node-sqlcipher\build\deps\sqlite3.vcxproj]
gyp ERR! build error 
gyp ERR! stack Error: `C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\MSBuild.exe` failed with exit 
code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:277:12)
gyp ERR! System Windows_NT 10.0.19042
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "build" "--fallback-to-build" "--module=C:\\repos\\node-sqlcipher\\lib\\binding\\napi-v6-win32-arm64\\node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=C:\\repos\\node-sqlcipher\\lib\\binding\\napi-v6-win32-arm64" "--napi_version=7" "--node_abi_napi=napi" "--napi_build_version=6" "--node_napi_label=napi-v6"
gyp ERR! cwd C:\repos\node-sqlcipher
gyp ERR! node -v v14.14.0
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --module=C:\repos\node-sqlcipher\lib\binding\napi-v6-win32-arm64\node_sqlite3.node --module_name=node_sqlite3 --module_path=C:\repos\node-sqlcipher\lib\binding\napi-v6-win32-arm64 --napi_version=7 --node_abi_napi=napi --napi_build_version=6 --node_napi_label=napi-v6' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (C:\repos\node-sqlcipher\node_modules\node-pre-gyp\lib\util\compile.js:83:29)  
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1048:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
node-pre-gyp ERR! System Windows_NT 10.0.19042
node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\repos\\node-sqlcipher\\node_modules\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd C:\repos\node-sqlcipher
node-pre-gyp ERR! node -v v14.14.0
node-pre-gyp ERR! node-pre-gyp -v v0.15.0
node-pre-gyp ERR! not ok 
Failed to execute 'C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --module=C:\repos\node-sqlcipher\lib\binding\napi-v6-win32-arm64\node_sqlite3.node --module_name=node_sqlite3 --module_path=C:\repos\node-sqlcipher\lib\binding\napi-v6-win32-arm64 --napi_version=7 --node_abi_napi=napi --napi_build_version=6 --node_napi_label=napi-v6' (1)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @journeyapps/[email protected] install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @journeyapps/[email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\denni\AppData\Roaming\npm-cache\_logs\2021-02-20T15_50_38_549Z-debug.log

C:\repos\node-sqlcipher>

I tried building node-sqlite3 locally, and it works. Running tests through npm test results in:

160 passing (16s)

... so that's a good start.

I don't know yet what exactly is going wrong, but will investigate later. Wanted to at least report my findings so far here 😊

Would love to get node-sqlcipher to build/run on Windows arm64 and have a device (Surface Pro X) at hand for testing.

[dennisameling]

Just tried building with this PR and that seems to work: #72

I can build node_sqlite3.node, but unfortunately Signal Desktop is still crashing. Will try to get a stack trace from Visual Studio soon.

[dennisameling]

Alright, have been able to dig a bit deeper, but this is where my knowledge of debugging/C stops. Would very much appreciate if someone could point me in the right direction as to what exactly is going wrong here 😊

I tried two things:

• Windows arm64 build using the 32-bit OpenSSL library (Windows ARM64 can run x64/32-bit executables through emulation)
• Windows arm64 build using native arm64 OpenSSL libraries I compiled from source (you can download them here if you want), based on this commit that Signal Desktop is using: EvanHahn-Signal@1691694

Windows arm64 build with 32-bit OpenSSL

Debug files which can be opened in the Visual Studio debugger (~532MB): https://fits4all-my.sharepoint.com/:u:/g/personal/dennis_fits4all_nl/EVKIMRKYSEZJqlvQk4ORy8gB07MXDwTR9_W0cLSygwpc_Q?e=IZcAox

Windows arm64 build with native arm64 OpenSSL libraries

Debug files which can be opened in the Visual Studio debugger (~506MB): https://fits4all-my.sharepoint.com/:u:/g/personal/dennis_fits4all_nl/ETyuNt0xdAZKmPKIKj2xW7oB8rHFkdqNaqcfaFgg0McF7w?e=dVvGQi

[rkistner]

Those are the same symptoms as I've seen for #67 - it crashes as soon as openssl is initialized. You should also note that the OpenSSL dll files are not listed as loaded in the dump.

The issue I had is that if you build against the Node runtime, it uses Node's OpenSSL symbols, which are not available when running on Electron. Instead, build for Electron directly. See the build arguments used here:

node-sqlcipher/.circleci/config.yml

Lines 22 to 25 in fe245cd

	if [[ "${ELECTRON_VERSION}" ]]; then
	GYP_ARGS="--runtime=electron --target=${ELECTRON_VERSION} --dist-url=https://electronjs.org/headers"
	fi
	./node_modules/.bin/node-pre-gyp rebuild package testpackage $GYP_ARGS

I'm guessing this is the issue in your case as well (although there could be more arm64-specific issues after this one is fixed).

[dennisameling]

That's super helpful info, thank you. Will try that later today 👍🏼

[dennisameling]

Alright, it works when I use statically built OpenSSL libraries that I added for ARM64 in this PR: https://github.com/EvanHahn-Signal/node-sqlcipher/pull/1/files

Would you accept a PR here that switches to statically linked OpenSSL for Windows (x86, x64 and arm64), @rkistner? That's what the Signal team did in their fork: EvanHahn-Signal@1691694 (libcrypto.lib, libssl.lib, ossl_static.pdb)

[rkistner]

Yes, it would be great to get rid of the dll's, and make the different platforms behave more consistently by statically linking everywhere.