Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(crypto): Initial support for SubtleCrypto #698

Merged
merged 66 commits into from
Dec 14, 2024

Conversation

nabetti1720
Copy link
Contributor

@nabetti1720 nabetti1720 commented Nov 23, 2024

Issue # (if available)

Closed #184

Description of changes

With this PR, we're making a small step forward with SubtleCrypto.
We're not yet fully web standards compliant, but we wanted to share our progress so far.

At the moment, the following functions are not implemented, but since it will be complicated and time-consuming from here, I would like to make it the next PR.

  • exportKey() w/o raw mode
  • importKey()
  • unwrapKey()
  • wrapKey()

Runtime compatibility (Results on a laptop):
image

Checklist

  • Created unit tests in tests/unit and/or in Rust for my feature if needed
  • Ran make fix to format JS and apply Clippy auto fixes
  • Made sure my code didn't add any additional warnings: make check
  • Added relevant type info in types/ directory
  • Updated documentation if needed (API.md/README.md/Other)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@nabetti1720 nabetti1720 marked this pull request as draft November 23, 2024 12:40
@Sytten
Copy link
Contributor

Sytten commented Nov 23, 2024

Nice!!!
Two comments:

  • We will need CryptoKey / CryptoKeyPair classes
  • Most methods need to be async as let spec and we will want to use tokio spawn_blocking I think.

@richarddavison
Copy link
Contributor

Super nice! Yeah, async needs to use ctx.spawn_exit(). For example see random_fill vs random_fill_sync in crypto module.

@Sytten
Copy link
Contributor

Sytten commented Nov 23, 2024

@richarddavison Not in this case I think, spawn_exit is only for running background tasks. Here we want to asyncify a blocking task, so you have to use the tokio thread pool for that. Same as when we do file operations.

@nabetti1720

This comment was marked as resolved.

@nabetti1720

This comment was marked as resolved.

Copy link
Contributor

@richarddavison richarddavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments

modules/llrt_crypto/Cargo.toml Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/decrypt.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/decrypt.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/decrypt.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/derive_bits.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/digest.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/Cargo.toml Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/sign.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/verify.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/verify.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
@nabetti1720

This comment was marked as resolved.

@Sytten
Copy link
Contributor

Sytten commented Nov 25, 2024

I would create a class to wrap the vec at the very least so the user cant assume it is an array. Then we can start implementing the props the spec requires.

@richarddavison
Copy link
Contributor

@richarddavison and @Sytten , Any good ideas on how to implement this? Is it possible to retain values ​​in LLRT that are not displayed in console.log?

Yes, create a Class for CryptoKey and CryptoKeyPair and keep hidden fields in rust only. Then expose only getters to what you need to provide as read only:

#[rquickjs::class]
#[derive(rquickjs::JsLifetime)]
struct CryptoKey<'js>{
   algorithm: Object<'js>,
   usages: Array<'js>,
   ...
}

impl<'js> CryptoKey<'js>{

    #[qjs(get)]
    pub fn algorithm(&self) -> Object<'js> {
        self.algorithm.clone()
    }
}

modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/Cargo.toml Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/decrypt.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/encrypt.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@richarddavison richarddavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic work so far! 🥇

modules/llrt_crypto/src/subtle/digest.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/generate_key.rs Outdated Show resolved Hide resolved
@richarddavison
Copy link
Contributor

This is great. Now we can probably add the CryptoTest from here https://github.com/web-platform-tests/wpt/tree/master

modules/llrt_crypto/src/subtle/crypto_key.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/crypto_key.rs Outdated Show resolved Hide resolved
@nabetti1720
Copy link
Contributor Author

nabetti1720 commented Nov 28, 2024

This is great. Now we can probably add the CryptoTest from here https://github.com/web-platform-tests/wpt/tree/master

Thank you. :) I'm always checking the test cases of wpt and looking for any unclear points in the implementation from the test cases. I'd like to incorporate it into the LLRT tests at some point.

However, it is unclear whether it conforms to the latest WebCryptoAPI specifications. For example, the generateKey test does not pass any required algorithm parameters other than name, which causes the test to fail.

https://github.com/web-platform-tests/wpt/blob/master/WebCryptoAPI/generateKey/successes.js

EDIT: I've looked at other test cases, but they don't seem to be compatible with the current SubtleCrypto interface. This is particularly fatal as the argument where the CryptoKey should be passed is an array. For now, I'll go ahead and create a minimal set of my own test cases.

@nabetti1720

This comment was marked as resolved.

@richarddavison
Copy link
Contributor

What's going on? I'm suddenly getting errors with Clippy, but they don't happen on my laptop...

image

Probably ci is using a newer version of clippy. Try upgrading.

@nabetti1720 nabetti1720 mentioned this pull request Nov 28, 2024
5 tasks
@nabetti1720

This comment was marked as off-topic.

Copy link
Contributor

@richarddavison richarddavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're getting closer 🎉

modules/llrt_crypto/src/subtle/mod.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/generate_key.rs Outdated Show resolved Hide resolved
modules/llrt_crypto/src/subtle/crypto_key.rs Outdated Show resolved Hide resolved
@Sytten
Copy link
Contributor

Sytten commented Dec 1, 2024

My point about the class CryptoKeyPair is you can make a rust class that is both readonly and the properties enumerable. IMO cleaner

#[rquickjs::class(frozen)] 
#[derive(Clone, Trace, rquickjs::JsLifetime)] 
pub struct CryptoKeyPair<'js> {
   #[qjs(get, enumerable)]
   private_key: Class<'js, CryptoKey<'js>>,
   #[qjs(get, enumerable)]
   public_key: Class<'js, CryptoKey<'js>>, 
}

@richarddavison
Copy link
Contributor

My point about the class CryptoKeyPair is you can make a rust class that is both readonly and the properties enumerable. IMO cleaner

#[rquickjs::class(frozen)] 
#[derive(Clone, Trace, rquickjs::JsLifetime)] 
pub struct CryptoKeyPair<'js> {
   #[qjs(get, enumerable)]
   private_key: Class<'js, CryptoKey<'js>>,
   #[qjs(get, enumerable)]
   public_key: Class<'js, CryptoKey<'js>>, 
}

Yes but the spec uses an object

@richarddavison
Copy link
Contributor

@nabetti1720 thanks for this fantastic PR! Hold off on more implementations according to spec as I'm doing some major refactoring. We can add more implementations in other PRs as this is already quite big :)

@nabetti1720
Copy link
Contributor Author

nabetti1720 commented Dec 7, 2024

@nabetti1720 thanks for this fantastic PR! Hold off on more implementations according to spec as I'm doing some major refactoring. We can add more implementations in other PRs as this is already quite big :)

@richarddavison , thank you for your attention to this PR. I agree. I'd like to think about implementing the remaining features after your refactoring is complete. Also, if possible, I would appreciate it if you could release commits little by little. I think not only me but everyone else is looking forward to seeing you complete this work. :)

@richarddavison
Copy link
Contributor

Hi @nabetti1720. Id try keeping changes to a minimum and go little by little but I don't want to commit things in a broken state. Since what I'm doing now is refactoring the fundamental enums and more applying SOLID principles there will be a lot of modifications. In summery I'm moving things to shared enums, moving methods to enums, implementing FromJs IntoJs traits where I can and reorganizing by applying stronger separation of concern. Also there was a lot of duplication which is now removed. What's left is derive and then I'll push

@richarddavison
Copy link
Contributor

Ok, I pushed what I have right now. This was a major rewrite, I'll try to summarize here:

  1. Divided signing and encryption params into separate enums convertible from JS
  2. Remove hardcoded SHA hashes, these are now derived from the key
  3. Moved a lot of logic to implement on the enums instead of in functions
  4. Reduced code duplication. Some enums have shared keys (KeyDerivation) etc
  5. Added more guardrails according to spec
  6. Verified that keygen + action algorithms matched.
  7. ...a lot more

What's left is fixing broken tests. I'm not a 100% sure we got the verification of usages right, for import they seem to break right now. There is also a big performance hit using RSA but that will be fixed once that crate switches some dependencies.

@nabetti1720
Copy link
Contributor Author

nabetti1720 commented Dec 13, 2024

Great refactoring! Let me know if there's anything else I should do next.
I have already confirmed on my laptop that the following issues occur after resolving the usage issue:

SubtleCrypto deriveBits/deriveKey > should be processing ECDH algorithm
Error: PKCS#8 ASN.1 error: unexpected ASN.1 DER tag: expected SEQUENCE, got OCTET STRING
    at <anonymous> (/Users/shinya/Workspaces/llrt/bundle/js/__tests__/unit/crypto.subtle.test.js:874:53)

SubtleCrypto deriveBits/deriveKey > should be processing HKDF algorithm
Error: PKCS#8 ASN.1 error: incorrect length for OCTET STRING
    at <anonymous> (/Users/shinya/Workspaces/llrt/bundle/js/__tests__/unit/crypto.subtle.test.js:1051:55)

SubtleCrypto deriveBits/deriveKey > should be processing PBKDF2 algorithm
Error: PKCS#8 ASN.1 error: unexpected ASN.1 DER tag: expected SEQUENCE, got OCTET STRING
    at <anonymous> (/Users/shinya/Workspaces/llrt/bundle/js/__tests__/unit/crypto.subtle.test.js:1236:55)

Indeed, this seems to be an error that occurs when using something that has been exported/imported.

EDIT: This is because while the conversion process when the exportKey is in raw mode is implemented, the reverse conversion process for the importKey in raw mode is not implemented.

@richarddavison
Copy link
Contributor

Great refactoring! Let me know if there's anything else I should do next. I have already confirmed on my laptop that the following issues occur after resolving the usage issue:

SubtleCrypto deriveBits/deriveKey > should be processing ECDH algorithm
Error: PKCS#8 ASN.1 error: unexpected ASN.1 DER tag: expected SEQUENCE, got OCTET STRING
    at <anonymous> (/Users/shinya/Workspaces/llrt/bundle/js/__tests__/unit/crypto.subtle.test.js:874:53)

SubtleCrypto deriveBits/deriveKey > should be processing HKDF algorithm
Error: PKCS#8 ASN.1 error: incorrect length for OCTET STRING
    at <anonymous> (/Users/shinya/Workspaces/llrt/bundle/js/__tests__/unit/crypto.subtle.test.js:1051:55)

SubtleCrypto deriveBits/deriveKey > should be processing PBKDF2 algorithm
Error: PKCS#8 ASN.1 error: unexpected ASN.1 DER tag: expected SEQUENCE, got OCTET STRING
    at <anonymous> (/Users/shinya/Workspaces/llrt/bundle/js/__tests__/unit/crypto.subtle.test.js:1236:55)

Indeed, this seems to be an error that occurs when using something that has been exported/imported.

EDIT: This is because while the conversion process when the exportKey is in raw mode is implemented, the reverse conversion process for the importKey in raw mode is not implemented.

Thanks! I'll fix usage checks according to this table:

┌─────────┬─────────────────────┬─────────────────────────────────────┬─────────────────────────────────────┬─────────────────────────────────────┬─────────────────────────────────────┐
│ (index) │ Algorithm           │ Generate Key                        │ Derive Key                          │ Import Key                          │ Required Usages                     │
├─────────┼─────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┤
│ 0       │ 'AES-GCM'           │ 'encrypt,decrypt,wrapKey,unwrapKey' │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │ 'encrypt,decrypt,wrapKey,unwrapKey' │
│ 1       │ 'AES-CBC'           │ 'encrypt,decrypt,wrapKey,unwrapKey' │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │ 'encrypt,decrypt,wrapKey,unwrapKey' │
│ 2       │ 'AES-CTR'           │ 'encrypt,decrypt,wrapKey,unwrapKey' │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │ 'encrypt,decrypt,wrapKey,unwrapKey' │
│ 3       │ 'AES-KW'            │ 'wrapKey,unwrapKey'                 │ '-'                                 │ 'wrapKey,unwrapKey'                 │ 'wrapKey,unwrapKey'                 │
│ 4       │ 'HMAC'              │ 'sign,verify'                       │ '-'                                 │ 'sign,verify'                       │ 'sign,verify'                       │
│ 5       │ 'Ed25519'           │ 'sign,verify'                       │ '-'                                 │ '-'                                 │ 'sign'                              │
│ 6       │ 'RSA-OAEP'          │ 'encrypt,decrypt,wrapKey,unwrapKey' │ '-'                                 │ '-'                                 │ 'decrypt,unwrapKey'                 │
│ 7       │ 'RSA-PSS'           │ 'sign,verify'                       │ '-'                                 │ '-'                                 │ 'sign'                              │
│ 8       │ 'RSASSA-PKCS1-v1_5' │ 'sign,verify'                       │ '-'                                 │ '-'                                 │ 'sign'                              │
│ 9       │ 'ECDSA'             │ 'sign,verify'                       │ '-'                                 │ '-'                                 │ 'sign'                              │
│ 10      │ 'ECDH'              │ 'deriveKey,deriveBits'              │ 'encrypt,decrypt,wrapKey,unwrapKey' │ '-'                                 │ 'deriveKey,deriveBits'              │
│ 11      │ 'HKDF'              │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │
│ 12      │ 'PBKDF2'            │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │
│ 13      │ 'X25519'            │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │ '-'                                 │ 'encrypt,decrypt,wrapKey,unwrapKey' │
└─────────┴─────────────────────┴─────────────────────────────────────┴─────────────────────────────────────┴─────────────────────────────────────┴─────────────────────────────────────┘

@richarddavison
Copy link
Contributor

@nabetti1720 we have two options to proceed here, either fix imports or disable imports completely, merge and open a new PR with this capability. This PR is already enormous 👍

@nabetti1720
Copy link
Contributor Author

nabetti1720 commented Dec 13, 2024

we have two options to proceed here, either fix imports or disable imports completely, merge and open a new PR with this capability. This PR is already enormous

@richarddavison , Thank you! As you say, this PR is already too big. I think it would be better to disable importKey first and merge it. After that, it would be better to open an issue that we currently know about and work towards resolving it.

I'm sorry to trouble you, but could you please merge this PR?

Alternatively, if you would like me to take measures to exclude the importKey (stop publishing it, exclude it from the test code), please let me know.

@richarddavison
Copy link
Contributor

Alternatively, if you would like me to take measures to exclude the importKey (stop publishing it, exclude it from the test code), please let me know.

@nabetti1720 please go ahead! We can just disable the export from the module and add a it.skip(...) or describe.skip(...) for the tests

@nabetti1720
Copy link
Contributor Author

@nabetti1720 please go ahead! We can just disable the export from the module and add a it.skip(...) or describe.skip(...) for the tests

@richarddavison , Instead of skipping all the tests, I kept what I could. All the tests should pass. We can finally see the goal. :)

tests/unit/crypto.subtle.test.ts Outdated Show resolved Hide resolved
@richarddavison richarddavison merged commit 5a03d95 into awslabs:main Dec 14, 2024
11 checks passed
@panva
Copy link

panva commented Dec 14, 2024

@nabetti1720 may i ask which tool presents the compatibility tables in the PR's description?

@nabetti1720
Copy link
Contributor Author

nabetti1720 commented Dec 14, 2024

may i ask which tool presents the compatibility tables in the PR's description?

Runtime compatibility - https://runtime-compat.unjs.io/

The source for this site is maintained on GitHub at:
https://github.com/unjs/runtime-compat

I cloned it from GitHub and ran it manually on my laptop to analyze it.

pnpm run --filter "*llrt-runtime" build && pnpm run --filter "*llrt-runtime" start && pnpm generate:process && pnpm run website

@nabetti1720 nabetti1720 deleted the feat/subtle-crypto branch December 14, 2024 13:00
@panva
Copy link

panva commented Dec 14, 2024

may i ask which tool presents the compatibility tables in the PR's description?

@richarddavison Runtime compatibility - runtime-compat.unjs.io

The source for this site is maintained on GitHub at: unjs/runtime-compat

Ah, so it's not based on WPTs. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SubtleCrypto
4 participants