Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Revisit read Txn and Lock table for read operations #33

Open
citrusraj opened this issue Dec 19, 2019 · 1 comment
Open

Revisit read Txn and Lock table for read operations #33

citrusraj opened this issue Dec 19, 2019 · 1 comment

Comments

@citrusraj
Copy link
Contributor

Currently read does not acquire locks due to life time of RDD is unspecified hence would lead to blocking of the InsertOverwrites. So protecting it against the DDL / Truncate / Compaction is operational issue as it stands.

Currently whenever DF materialises first time the transactional snapshot of read is acquired at that time.

Above two need rethinking ..
@citrusraj
Copy link
Contributor Author

2e66f76

This commit adds frame work to perform operation under transactions.

sourabh912 pushed a commit to sourabh912/spark-acid that referenced this issue Jul 31, 2020
Merged in SPAR-4598 (pull request qubole#33)
fe5835c 
fix: dev: SPAR-4598: Performance improvements in acid writer.

AcidWriter process every InternalRow one by one in process() method. We identified few places in the code flow where redundant objects were being created, which were slowing down the write performance.
Approved-by: Amogh Margoor <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@citrusraj