“Try Valkey” - An In-Browser Valkey Server and CLI

[zarkash-aws]

Abstract:

This proposal introduces a web-based environment for Valkey customers to interact with Valkey-CLI’s commands. The project aims to provide an intuitive and interactive platform for exploring Valkey's functionalities without requiring local setup or additional server infrastructure.

Motivation:

New users often prefer to evaluate a tool before committing to a full download and installation, but setting up a local environment or server infrastructure for testing can be complex and time-consuming. This feature aims to solve this by providing an easy-to-use, in-browser Valkey CLI that requires no local setup, enabling users to quickly explore Valkey’s capabilities. By lowering the entry barrier, this solution will encourage more users to try Valkey and ultimately increase adoption.

Design Considerations:

We aimed for the project to meet the following design considerations:

1. Valkey CLI Experience: Allow users to interact with core Valkey commands directly in the browser, much like the Valkey CLI tool.
2. Frugality: Minimize ongoing maintenance effort and resource consumption, ensuring the project remains efficient and sustainable over time.
3. Efficiency: Optimize the solution for smooth performance, ensuring it runs efficiently without requiring substantial local resources, while delivering a responsive web app experience.

Specification:

The proposed solution leverages a v86 virtual machine (VM) that operates entirely within the web browser. This approach is designed to be lightweight, flexible, and entirely self-contained.
Try it out: https://zarkash-aws.github.io/try-valkey.github.io/

In-Browser Execution

• All components, including the Valkey CLI and server, run directly within the user's browser. There is no backend server, ensuring the solution is self-contained and does not require additional infrastructure.

Preconfigured VM Image

• The initial image is based on Alpine Linux, a lightweight distribution that is ideal for minimizing resource consumption.
• The image is created using a custom Dockerfile and preconfigured with Valkey Server version 7.2.6 (32-bit) and the Valkey CLI, installed via Alpine's package manager (apk). The state of the image is such that the Valkey server is already loaded, and the Valkey CLI is running on serial port 0. This ensures that the user can immediately interact with the Valkey CLI upon loading the state.
• The image files are simple to host and serve, aligning with the goal of creating a lightweight, infrastructure-free tool. The image is served in a compressed format, reducing load times and optimizing resource usage. Importantly, the VM runs entirely within the user's local browser environment upon opening the webpage, ensuring that all operations occur locally without relying on external servers.

The VM's reliance on a preconfigured image allows for a variety of flexible configurations:

• Different Valkey Versions: Users can experiment with specific versions to assess compatibility or explore features. This can be achieved by creating distinct images for each version, with the appropriate image loaded into the VM.
• Tutorial Scenarios: Preconfigured VM states can guide users through various Valkey functionalities, such as initialized datasets or simulated clusters. These states can be captured by freezing the VM at specific points after loading the server with predefined keys.
• Advanced Use Cases: The VM is capable of simulating more complex scenarios, such as multi-server cluster setups. Since the environment is virtualized, multiple servers can be run within the VM for such simulations.

UX/UI:

• Users interact with the Valkey CLI through a browser-based terminal powered by xterm.js. The CLI’s input and output are routed through the VM’s serial port 0 (ttyS0), ensuring seamless communication with the virtualized environment.

Other Alternatives We’ve Considered:

1. Backend Server: A backend server was considered, but it introduced challenges such as managing infrastructure, ensuring that data is properly loaded for each client (without one client inadvertently accessing data already loaded by another), and handling multiple clients connecting simultaneously. This approach also raises security concerns, as sensitive data could be exposed due to centralized server storage. Additionally, it would require ongoing resources for server maintenance, scaling, and security management, resulting in higher costs and complexity over time.
2. Emulating the CLI: We considered simulating the Valkey CLI experience in the browser by faking the interactions. However, this approach was deemed impractical as it would have required substantial development effort and ongoing maintenance. The complexity of accurately replicating the CLI behavior without full functionality would have led to a less reliable solution.
3. WASM: Another option considered was compiling Valkey to WebAssembly (WASM). However, since WASM does not support TCP networking which the Valkey server and CLI rely on, we needed to implement a workaround. A potential solution involved using WebRTC with ICE candidates for networking, which required implementing a new connection type that supports WebRTC logic. While technically feasible, this approach proved complex and difficult to maintain due to the need for updates and modifications to Valkey’s networking logic. Additionally, the development time needed to implement and maintain this approach was considerably higher. Ultimately, the decision was made to use a virtualized solution for better flexibility and maintainability.

Workflows:

To keep the in-browser environment up to date with new versions of Valkey, we plan to automate the image creation whenever a new Valkey version is released. This can be achieved similarly to how Docker images are updated. When a new major, minor, or patch release of Valkey is available, the release workflow will automatically trigger the creation of a new VM image. This process will follow a pattern similar to the Docker update, where the relevant commits are merged, and versioning information is updated. Once the new version is released, the workflow will automatically trigger the creation of a new VM image, which will include the updated Valkey server and CLI. The image will then be compressed, and made available for use in the web-based environment, ensuring that the latest version of Valkey is ready for users in the browser.

Open Issues:

• remove cookies / add a consent message
• prevent ctrl+c from terminating the server
• automatically shutdown emulator if it's idle for a long time
• version automation

[ranshid]

@valkey-io/core-team + @stockholmux this projects is aiming at introducing a great utility to be used in our website, but having it automatically generating images for web embedding for each release is also planned. can you take a look (and please play with the linked demo, it is sooooo cool :) ) and provide your feedback?

[zuiderkwast]

🌩️
🎩
😲

[zuiderkwast]

This is incredible! Let's put it on the start page of the website! 🍭

Automated builds can come later.

[stockholmux]

😍 This is pretty slick.

I did get a sneak peak at the proof of concept and this has progressed extremely fast from that, especially with the UI/UX which is now totally usable.

A few questions @zarkash-aws and @ranshid:

• Why 32-bit Valkey?
• Why 7.2.x not 8.0.x?

One minor nit we'll need to clear up - looks like it's installing some cookies. I suspect these are from CDN assets or something but we've tried to keep valkey.io a cookie-free zone (for GDPR and other reasons).

We'll also need to document this tool well because I'm sure it has hundreds of uses.

[hwware]

It is an amazing web demo, I like to play it, but I have 2 small questions:

1. How multiply clients can work together?
2. How we can use memtier-benchmark or valkey-benchmark to play it?
3. When I press Ctrl+C in the client, it can not connect to server again? Is there a way to resume it?

Anyway, great work Thanks

[zuiderkwast]

One minor nit we'll need to clear up - looks like it's installing some cookies. I suspect these are from CDN assets or something but we've tried to keep valkey.io a cookie-free zone (for GDPR and other reasons).

Just a comment on this. We try to avoid cookies, but if the demo needs it, we can just mention that in the "press any key to start" message, that you consent by using it. GDPR doesn't require an annoying banner for legitimate cookies required for the functionality (such as login session cookie when a user logs in). The annoying banner is just for illegitimate activities like user tracking.

[ranshid]

Why 32-bit Valkey?

The project we are building on currently only supports X86 as it emulates a PENTIUM 4 cpu using WASM and not WASM64. AFAIK there are no current plans by the project owners to support 64bit emulator, but that is something we can look for in the future.

Why 7.2.x not 8.0.x?

That was just for the demo. IMO in the future we will create automation to release new image with every valkey version release, much like the process we take for containers release.

How multiply clients can work together?

Currently we only created a single client. It is possible to create multiple clients and have them run on different tty's I suppose, but we will have to check that.

How we can use memtier-benchmark or valkey-benchmark to play it?

Since this emulator runs OS (Alpine in this case) it is possible to run any tool we want which would operate on this 32bit OS (ubuntu is also possible). However I do not think this is meant for performance benchmarking but mainly as a sandbox to experience with new featuresand commands.

When I press Ctrl+C in the client, it can not connect to server again? Is there a way to resume it?

Yes. CTRL-C currently also bring down the server. we will have to fix that (it pains me when testing subscribe in-and-out mode). @zarkash-aws lets manage an open issues list in the top comment.

[zuiderkwast]

My browser was killed by the OS because it was consuming too much memory over several hours. I had the the "try valkey" in a tab. Is it eating memory? If it does, then perhaps we should make it shut down automatically if it's idle for a long time, like hours.

[hwware]

Why 32-bit Valkey?

The project we are building on currently only supports X86 as it emulates a PENTIUM 4 cpu using WASM and not WASM64. AFAIK there are no current plans by the project owners to support 64bit emulator, but that is something we can look for in the future.

Why 7.2.x not 8.0.x?

That was just for the demo. IMO in the future we will create automation to release new image with every valkey version release, much like the process we take for containers release.

How multiply clients can work together?

Currently we only created a single client. It is possible to create multiple clients and have them run on different tty's I suppose, but we will have to check that.

Thanks!!

How we can use memtier-benchmark or valkey-benchmark to play it?

Since this emulator runs OS (Alpine in this case) it is possible to run any tool we want which would operate on this 32bit OS (ubuntu is also possible). However I do not think this is meant for performance benchmarking but mainly as a sandbox to experience with new featuresand commands.

When I press Ctrl+C in the client, it can not connect to server again? Is there a way to resume it?

Yes. CTRL-C currently also bring down the server. we will have to fix that (it pains me when testing subscribe in-and-out mode). @zarkash-aws lets manage an open issues list in the top comment.

Thanks for your great work. 

[ranshid]

My browser was killed by the OS because it was consuming too much memory over several hours. I had the the "try valkey" in a tab. Is it eating memory? If it does, then perhaps we should make it shut down automatically if it's idle for a long time, like hours.

@zuiderkwast which browser did you use? for example my chrome can basically consume so much memory doing nothing my computer sometimes go into SWAP :)

[zuiderkwast]

Firefox. Maybe I have swap disabled, I don't know. :D It might be another tab that consumed memory. I have no proof.

[ranshid]

@zarkash-aws lets take this action item - sounds important to handle if correct

[madolson]

@ranshid We discussed on another thread, but I think the next step is to move this project to its own repo that uploads it to our asset s3 bucket (you can talk to @roshkhatri who set it up for this repo) and then open a PR with rendering the site on the main website. Do you have any concerns with that?

[ranshid]

@ranshid We discussed on another thread, but I think the next step is to move this project to its own repo that uploads it to our asset s3 bucket (you can talk to @roshkhatri who set it up for this repo) and then open a PR with rendering the site on the main website. Do you have any concerns with that?

Correct this is currently the effort @zarkash-aws is working on