fix(form): optimize calculated question performance #2339
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
czosel
commented
Dec 2, 2024
•
czosel
commented
- drop several signals in favor of explicit calls in domain logic to allow reuse of form structure
- add prefetching for document structure
- apply topological sorting before calculating answers on document creation
|
@czosel Note: Instance creation is still slow, and seems to be triggered by create in SaveDocumentLogic. I'm having a look. |
|
Form changes (e.g. through form-builder) are probably also still slow. |
This sorts the graph that is formed by calculated questions and their dependents, and then initializes them in the correct order.
czosel
force-pushed
the
fix-calculated-question-performance
branch
4 times, most recently
from
a47c6e1 to
33746a0
Compare
czosel
changed the title
czosel
force-pushed
the
fix-calculated-question-performance
branch
2 times, most recently
from
4584728 to
d1bd793
Compare
czosel
commented
Dec 31, 2024
Comment on lines
-104
to
-122
| @receiver(post_save, sender=models.Question) | ||
| @disable_raw | ||
| @filter_events(lambda instance: instance.type == models.Question.TYPE_CALCULATED_FLOAT) | ||
| def update_calc_from_question(sender, instance, created, update_fields, **kwargs): | ||
| # TODO optimize: only update answers if calc_expression is updated | ||
| # needs to happen during save() or __init__() | ||
|
|
||
| for document in models.Document.objects.filter(form__questions=instance): | ||
| update_or_create_calc_answer(instance, document) | ||
|
|
||
|
|
||
| @receiver(post_save, sender=models.FormQuestion) | ||
| @disable_raw | ||
| @filter_events( | ||
| lambda instance: instance.question.type == models.Question.TYPE_CALCULATED_FLOAT | ||
| ) | ||
| def update_calc_from_form_question(sender, instance, created, **kwargs): | ||
| for document in instance.form.documents.all(): | ||
| update_or_create_calc_answer(instance.question, document) |
There was a problem hiding this comment.
I dropped these two handlers completely. On my local instance, updating a calculated question took about 10s, even though I only had 10 documents in the DB. Since in typical prod scenarios we have thousands of documents, this kind of thing should be done, if needed, explicitly in a management command.
czosel
force-pushed
the
fix-calculated-question-performance
branch
from
d1bd793 to
ad4d2e7
Compare
czosel
commented
Dec 31, 2024
| calc_question.calc_expression = "'sub_question'|answer -1" | ||
| calc_question.save() | ||
| calc_ans.refresh_from_db() | ||
| assert calc_ans.value == 99 |
There was a problem hiding this comment.
This functionality was dropped: When updating the calc expression of a calculated question, the answers are not updated automatically anymore.
czosel
force-pushed
the
fix-calculated-question-performance
branch
from
ad4d2e7 to
55cf7ec
Compare
czosel
force-pushed
the
fix-calculated-question-performance
branch
from
0426441 to
af7e014
Compare
czosel
commented
Dec 31, 2024
| ], | ||
| ) | ||
| ) # fmt:skip |
There was a problem hiding this comment.
drive-by fix: exclude this block from autoformatting so that all parametrizations can stay one one line
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.