Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Explorer in VS Code Extension and @Test() Attribute #2059

Open
wants to merge 42 commits into
base: main
Choose a base branch
from

Conversation

sezna
Copy link
Contributor

@sezna sezna commented Dec 10, 2024

This PR is for enabling VS Code's Test Explorer functionality in our extension, and it introduces the @Test() attribute on callables. It also switches our libraries over to use @Test() instead of the old Main-based pattern.

Any callable that takes no arguments can be annotated as a @Test(). If it does not crash (via Fact or otherwise), it is considered a pass. If it does crash, then it is considered a failure.

After this PR is in, when you open a .qs file, you'll see this icon:
image

Clicking this icon will take you to the test explorer:
image

The test explorer shows all discovered tests in their namespace hierarchy. Clicking the "run" play button on a namespace runs all child tests contained in that namespace. A test turns green if its last run passed, or red if it didn't.

Other features:

"Run Test" button next to the test itself

image

"Failed test" icon on failed tests

image

Notes:

  1. This PR also fixes a bug where the Q# event handler would print [object Object] as the output of a failed run.
  2. This PR adds an event emitter to the vscode extension. This event emitter is used to emit events any time updateDocument is triggered. The test explorer listens for this event to refresh tests for a specific URI. The test discovery code has to scan for tests across an entire package, since tests could be added to a namespace from any file due to explicit namespaces. But only the updates from the new URI are actually populated to the test explorer.

Potential future work:

  • Support test cancellation tokens
  • Support "continuous mode" where we auto-rerun tests based on watching for file changes
  • Test coverage tracking (VS Code supports tracking lines of code coverage through the test explorer)
  • Debug/Coverage profile modes (currently, only the "Run" profile is supported)
  • A "run test" codelens on callables annotated with @Test() (this was deferred for now as I want that experience to integrate with the Debugger service and that requires a bit more work).

vscode/src/testExplorer.ts Fixed Show fixed Hide fixed
vscode/src/testExplorer.ts Fixed Show fixed Hide fixed
vscode/src/testExplorer.ts Fixed Show fixed Hide fixed
vscode/src/testExplorer.ts Fixed Show fixed Hide fixed
vscode/src/testExplorer.ts Fixed Show fixed Hide fixed
@sezna sezna marked this pull request as ready for review December 12, 2024 21:43
github-merge-queue bot pushed a commit that referenced this pull request Dec 22, 2024
This was a drive-by update from my work in #2059. Just wanted it in a
separate PR to keep the noise down.

It seems to have triggered some `prettier` fixes too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants