Auto-generate JSON Schema from TypeScript

[alexiswl]

Summary

1. Run typescript-json-schema over the Workflow schema.
2. Refine the schema (see .github/scripts/curate_autogenerated_schema.py)

Schema refining process steps

1. Remove loadingOptions and extensionFields properties

2. Rename class_, type_ etc to class, type

3. Add $import and $include options and update requirements that allow for them (SchemaDefRequirementa and InlineJavascriptRequirement)

4. Fix inputs / outputs / steps, so not just an array but can be an object where the key is the id attribute.

5. Fix Requirements / Hints, so not just an array but can be an object where the key is the class attribute.

6. Fix descriptions by removing Auto-generated... from the start

7. Generate the CWLFile definition, which can be one of Workflow, CommandLineTool, or ExpressionTool and also fits the CWLMetadata

8. Fix issues with additionalProperties and nested definitions as shown here https://stoic-agnesi-d0ac4a.netlify.app/1

9. Write out the schema

10. Generate the CWLGraph definition, which is an array of CWLFile objects with CWLMetadata

11. Write out the graph schema to a separate file

[alexiswl]

Installation

JetBrains

Settings -> Languages & Frameworks -> Schemas and DTDs -> JSON Schema Mappings:

Set Schema file or URL to https://github.com/alexiswl/cwl-ts-auto/blob/enhancement/generate-json-schema/cwl_schema.json

Set Schema Version to 'JSON Schema version 7'

Click Apply

VSCode

1. Install the RedHat YAML Extension (this will enable the yaml.schemas key in the settings.json

2. Open the settings json file and paste in the following keys

The settings.json can be accessed via the command pallete.

We also need to update the files.associations so that .cwl files are seen as YAML files by the compiler.

{
    "yaml.schemas": {
        "https://raw.githubusercontent.com/alexiswl/cwl-ts-auto/enhancement/generate-json-schema/cwl_schema.json": "*.cwl"
    },
    "files.associations": {
        "*.cwl": "yaml"
    }
}

Uses

Highlight invalid keys

Show documentation

Provide Valid Suggestions

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (0c14cfc) 55.62% compared to head (08dee13) 55.62%.

❗ Current head 08dee13 differs from pull request most recent head 7bdbfd7. Consider uploading reports for the commit 7bdbfd7 to get more accurate results

Additional details and impacted files

@@                              Coverage Diff                               @@
##           enhancement/auto-generate-json-schema-from-ts      #27   +/-   ##
==============================================================================
  Coverage                                          55.62%   55.62%           
==============================================================================
  Files                                                127      127           
  Lines                                               7837     7837           
  Branches                                            1572     1572           
==============================================================================
  Hits                                                4359     4359           
  Misses                                              3111     3111           
  Partials                                             367      367           

Flag	Coverage Δ
unittests	55.62% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[alexiswl]

Ah makes sense, this is a cross-fork write out error.

Can we merge this into a branch on the origin repo first and then retry the GH action test? I don't think I have permission to make a branch on this repository

[mr-c]

@alexiswl I sent you an invitation to join https://github.com/orgs/common-workflow-lab/teams/people/members which gives you permission to make branches in this and many other CWL repos. Feel free to re-open this PR from a local branch and thanks again for your contributions!

[mr-c]

Are there any advantages to producing a schema using the newer drafts of the JSON schema spec?

[alexiswl]

Potentially, JetBrains only seems to support version 7 though from the drop down menus, but I don't know if they actually support more recent versions (2019 and 2020). From the JSON Schema Store, GitHub Actions schema is also on version 7 still

[alexiswl]

Updated base branch to equivalent branch name on the common-workflow-lab/cwl-ts-auto repo.

Merging and will then create the PR to main from the common-workflow-lab branch instead.