Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring parser tests #1237

Merged
merged 8 commits into from
Nov 8, 2023
Merged

Refactoring parser tests #1237

merged 8 commits into from
Nov 8, 2023

Conversation

konnov
Copy link
Contributor

@konnov konnov commented Nov 7, 2023

Towards #17. I have carved out an important piece of refactoring from #1221: Instead of testing error messages via the JSON output, test them directly in unit tests. This significantly improves our parser tests:

  • It is clear what code is being parsed. One does not have to look into a separate file.
  • The error message is clearly specified. Again, we do not have to look into a JSON file (nobody does that).
  • It's harder to ignore the failing tests, that is, there is no automatic way of agreeing with failing tests (as is happening with our current script).
  • We are testing the error messages, instead of testing the JSON format. There should be but a few tests that actually test the JSON output.

I have only refactored the tests of phase 1. We should understand whether we want to refactor the remaining phases the same way.

@konnov konnov requested review from bugarela and shonfeder November 7, 2023 13:48
@konnov
Copy link
Contributor Author

konnov commented Nov 7, 2023

@bugarela, this test fails in a weird way. I guess, it is the result of recent refactoring. What shall we do about it?

     Error: Value is None
      at errorFactory (node_modules/@sweet-monads/maybe/cjs/index.js:105:33)
      at MaybeConstructor.unwrap (node_modules/@sweet-monads/maybe/cjs/index.js:108:15)
      at ToIrListener.exitConst (src/parsing/ToIrListener.ts:118:36)
      at ConstContext.exitRule (src/generated/QuintParser.ts:4074:13)
      at ParseTreeWalker.exitRule (src/tree/ParseTreeWalker.ts:105:7)
      at ParseTreeWalker.walk (src/tree/ParseTreeWalker.ts:57:11)
      at parsePhase1fromText (src/parsing/quintParserFrontend.ts:120:27)
      at parseErrorsFrom (test/parsing/quintParserFrontend.test.ts:36:29)
      at Context.<anonymous> (test/parsing/quintParserFrontend.test.ts:150:20)
      at processImmediate (node:internal/timers:478:21)

@konnov konnov self-assigned this Nov 7, 2023
@konnov konnov added refactoring parser Quint parser labels Nov 7, 2023
@bugarela
Copy link
Collaborator

bugarela commented Nov 7, 2023

@bugarela, this test fails in a weird way. I guess, it is the result of recent refactoring. What shall we do about it?

This is the same problem I've been mentioning, that should be fixed by #1228. I'm working on the debug error messages for that and then we can merge it (I believe), thus unblocking this. Alternatively, we can comment out the invalid const test and see if that's enough to make the error go away for now, and then I uncomment that test on my PR.

@bugarela
Copy link
Collaborator

bugarela commented Nov 7, 2023

@konnov I merged my PR and updated this branch, and the error is gone. Now there is only a mismatch in the error message.

@konnov
Copy link
Contributor Author

konnov commented Nov 8, 2023

@konnov I merged my PR and updated this branch, and the error is gone. Now there is only a mismatch in the error message.

thanks @bugarela! I have fixed the remaining unit test. Could you have a look. Then I would be able to faster iterate on #1227.

Copy link
Collaborator

@bugarela bugarela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is such a great improvement on readability and maintainability, thanks! It does make the tests weaker, and I'm specially concerned about testing if we are highlighting the correct portion of the code. Perhaps we can use the reportError facility and then compare the text with ^^ symbols, instead of looking at the locs.

Anyway, I think this is much better than what we had before and we can iterate on further improvements later!

describe('syntax errors', () => {
it('unbalanced module definition', () => {
const code = 'module empty {'
const errors = parseErrorsFrom('module-unbalanced', code)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make this source name parameter fixed so a value? I don't see why we need to name the source name for every test differently

quint/test/parsing/quintParserFrontend.test.ts Outdated Show resolved Hide resolved
@konnov konnov enabled auto-merge (squash) November 8, 2023 12:58
@konnov konnov merged commit 8772f56 into main Nov 8, 2023
15 checks passed
@konnov konnov deleted the igor/parser-tests-refactor branch November 8, 2023 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parser Quint parser
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants