Aya JavaScript Runtime

[nick-paul]

TL;DR: This PR creates a JS build of aya using maven (#106) and TeaVM. You can try the demo here. The editor can be used directly in documentation or examples

The demos above come in two parts:

1. The Aya JavaScript Runtime (this PR): Compile aya.js, a library that exposes a function aya.runIsolated(string) that takes aya source code and produces a result.
2. The Editor: The editor (and the rest of the demo site) is implemented in plain JS and markdown. The code lives here for now.

Even though the editor is not directly a part of this PR, it has many cool features so lets cover that first.

The Editor

Features

You can write and run aya code directly in your browser. The editor is based off of CodeMirror and has a bunch of cool features already such as

Syntax highlighting & standard library support:

Documentation on hover:

Linting:

Dark Mode:

Editor Details

The editor is implemented as a react component using CodeMirror. For the demo, the editor is embedded in a docusaurus project.

Aya Runtime (This PR)

The aya runtime is a custom aya library built using TeaVM. The aya.js runtime is relatively lightweight and loads quickly. However, execution is pretty slow if you are doing anything serious and I have already run into a few bugs (see below).

What works in the JS runtime:

• All "core" language features and operators
• Basic support for a virtual file-system so you can import the standard library

What does not work yet:

• Many extended operators :{...} are not supported yet. Some of them should be easy to support while others may take a bit more work.
• Notably, the graphics operators are not supported at the moment. The TeaVM site has examples with graphics so we should be able to support the graphics instructions.
• There are a handful of bugs (see below)

Virtual IO and Filesystem

In order to build this runtime, I had to remove any system, I/O, or filesystem dependencies from the build. My solution was to create an abstract class for filesystem and I/O operations. The standard desktop implementation uses an implementation that exposes the standard file and IO functions. The web version uses an implementation that creates a virtual filesystem and I/O. Full disclosure: this solution is a messy hack at the moment and needs to be cleaned up and thoroughly tested before merging.

The other benefit of using a virtual filesystem is that we can expose library functions to allow JavaScript to create and read files. Currently aya.js exposes a function addFile that takes a path and a string and creates a file available to aya. This is how the standard library is loaded.

To load the standard library, there is an aya script that reads all standard library files and writes them to a JSON object (aya-stdlib.js). The JSON object is read in by the JavaScript host which calls aya.addFile on each item.

Maven

Warning: This uses an old version of #106 and also need to be cleaned up. I was just focued on getting anything working for the first pass. I configured maven to build the JS runtime separate from the desktop runtime using build profiles. Use the following to build the JS version:

mvn install -Pweb

Known Bugs

Overall, I'm pretty happy with how well everything works. There are some strange new bugs with TeaVM that break some of the examples though:

This should throw an index error but returns NaN:

aya(JS)> [1 2 3].[100]
NaN

aya(Java)> [1 2 3].[100]  
Invalid index 100 for list [ 1 2 3 ]

Runtime exception with J

aya(JS)> [ 1 ] [ ] J
java.lang.RuntimeException: (JavaScript) TypeError: c is undefined

aya(Java)> [ 1 ] [ ] J
[ 1 ]

Runtime exception using .= on strings

aya(JS)> [1 2 3] [3 2 3] .=
[ 0 1 1 ] .# OK

aya(JS)> "abc" "dbc" .=  
java.lang.RuntimeException: (JavaScript) TypeError: c.jF is not a function

aya(Java)> "abc" "dbc" .=  
[ 0 1 1 ]

TODO

• Update to latest Add Maven build system #106
• Clean up filesystem and IO implementation
• Clean up maven
• Determine which extended operators can be supported

[nick-paul]

I think I'm happy with where this implementation is now. It is still quite limited compared to the desktop version so I think it is worth noting what the goals of this runtime are:

Goals

• 1. Aya "pad": The primary goal of this implementation is to support a lightweight aya scratchpad that is convient and easy to understand for new users.
  • Many popular languages have interactive repls on thier home page for new users to try:
    • BQN
    • Uiua
    • Janet
  • The web implementation has minimal support for linting and enhanced documentation (on hover) to also help new users learn the language. Other small web-specific enhancements may be made to make the interface as easy to use for new users as possible
• 2. Interactive Documentation: The secondary goal of this implementation is to support interactive documentation. I'd like to put this editor directly into the manual to users can try running examples directly on the page
  • The excellent Janet for Mortals book has a REPL built into the web page for testing examples
  • Go has interactive tutorials on their website
  • Many others...

Not Goals

• Feature Parity with Desktop: This runtime will never have all the features of the base language. Many features (such as graphics) are not included. This runtime is meant to be embedded into web pages using any front end such as an editor or directly in a documentation page. I spent some time exploring to see what it would take to have more feature parity but many extensions such as graphics and user interaction would require complete rewrites to be compatible with the JS backend supported by TeaVM.
  • I would still love to have a fully featured web IDE but it would likely be its own project or page on the site. It would also likely need to use a different VM like (CheerpJ which would allow drop-in running on the web). Something like @BlazingTwist s AyaWeb would be much better suited as a fully featured web IDE

What next?

After we merge the main change in, I'll be working through updating the manual using this repo. I'll move it over to github.com/aya-lang. Eventually we should have a nice new aya website with interactive documentation and an Aya "pad" so new users can try out the language.

@BlazingTwist, If you have a chance, can you take a look at the pom.xml and let me know if there is a better way to set things up? I split the build into two profiles because the standard build cannot have the teavm stuff in it (see this line). Otherwise the builds are pretty much the same. And if we don't actually need a profile, we could just always build the JS stuff in the package stage. By default, mvn ... builds the desktop profile which is the same as the current behavior on master. To build the web stuff (aya.js and aya-stdlib.js), you can use mvn package -Pweb and then test it by opening index.html in your browser.

(Also worth noting that I'm playing around with #111 on the side as well. I'm super excited to try building a package manager for aya that allows for using custom .jars)

[BlazingTwist]

Also the changes from #103 have gone missing (during a merge conflict maybe?) which caused 'package-ayastdlib-js' to fail for me.

I'm also getting a warning: [WARNING] Parameter 'mainPageIncluded' is unknown for plugin 'teavm-maven-plugin:0.10.2:compile (default)'
Odd, since the teaVM docs include the parameter as well.

I've tinkered with the pom for a bit and got it to run both profiles by keeping teavm / the web classes during compilation and excluding them from the fat jar instead.
PRs: #119 OR #120

Personally, I like the added configurability of the profiles.
However that might introduce unwanted complexity.

[BlazingTwist]

Seems input is never defined.

[nick-paul]

Thanks for the PR with the pom cleanup, this branch looks all good to me!