Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hivemapper model created #5997

Merged
merged 24 commits into from
Jun 3, 2024
Merged

hivemapper model created #5997

merged 24 commits into from
Jun 3, 2024

Conversation

alexus-98
Copy link
Contributor

Thank you for contributing to Spellbook!

Thank you for taking the time to submit code in Spellbook. A few things to consider:

  • If you are a first-time contributor, please sign the CLA by copy & pasting exactly what the bot mentions in PR comment
  • Refer to docs section below to answer questions
  • Dune team will review submitted PRs as soon as possible

Best practices

To speed up your development process in PRs, keep these tips in mind:

  • Each commit to your feature branch will rerun CI tests (see example)
    • This includes all modified models on your branch
    • This includes all history of the data
  • Two tips for faster development iteration:
    • Ensure dbt is installed locally (refer to main readme) and run dbt compile
      • This will output raw SQL in target/ directory to copy/paste and run on Dune directly for initial query testing
    • Hardcode a WHERE filter for only ~7 days of history on large source tables, i.e. ethereum.transactions
      • This will speed up the CI tests and output results quicker -- whether that's an error or fully successful run
      • Once comfortable with small timeframe, remove filter and let full history run

Incremental model setup

  • Make sure your unique key columns are exactly the same in the model config block, schema yml file, and seed match columns (where applicable)
  • There cannot be nulls in the unique key columns
    • Be sure to double check key columns are correct or COALESCE() as needed on key column(s), otherwise the tests may fail on duplicates

🪄 Use the built CI tables for testing 🪄

Once CI completes, you can query the CI tables and errors in dune when it finishes running.

  • For example:
    • In the run initial models and test initial models, there will be a schema that looks like this: test_schema.git_dunesql_4da8bae_sudoswap_v2_base_pool_creations
    • This can be temporarily queried in Dune for ~24 hours

Leverage these tables to perform QA testing on Dune query editor -- or even full test dashboards!

Spellbook contribution docs

The docs directory has been implemented to answer as many questions as possible. Please take the time to reference each .md file within this directory to understand how to efficiently contribute & why the repo is designed as it is 🪄

Example questions to be answered:

Please navigate through the docs directory to find as much info as you can.

Note: happy to take PRs to improve the docs, let us know 🤝

Copy link

github-actions bot commented May 26, 2024

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@dune-eng
Copy link

Workflow run id 9243750576 approved.

@alexus-98
Copy link
Contributor Author

I have read the CLA Document and I hereby sign the CLA

github-actions bot added a commit that referenced this pull request May 26, 2024
@dune-eng
Copy link

Workflow run id 9244271872 approved.

@alexus-98 alexus-98 marked this pull request as ready for review May 26, 2024 18:59
@dune-eng
Copy link

Workflow run id 9245736027 approved.

@dune-eng
Copy link

Workflow run id 9245736040 approved.

@dune-eng
Copy link

Workflow run id 9251998966 approved.

@dune-eng
Copy link

Workflow run id 9251999188 approved.

@dune-eng
Copy link

Workflow run id 9252141107 approved.

@dune-eng
Copy link

Workflow run id 9252141564 approved.

@dune-eng
Copy link

Workflow run id 9252301861 approved.

@dune-eng
Copy link

Workflow run id 9252301957 approved.

@dune-eng
Copy link

Workflow run id 9252448910 approved.

@dune-eng
Copy link

Workflow run id 9252449101 approved.

@dune-eng
Copy link

Workflow run id 9252665254 approved.

@dune-eng
Copy link

Workflow run id 9252665515 approved.

@dune-eng
Copy link

Workflow run id 9252986128 approved.

@dune-eng
Copy link

Workflow run id 9289083611 approved.

@dune-eng
Copy link

Workflow run id 9291287167 approved.

@dune-eng
Copy link

Workflow run id 9291287387 approved.

@jeff-dude jeff-dude added the WIP work in progress label May 29, 2024
@dune-eng
Copy link

Workflow run id 9301660494 approved.

@dune-eng
Copy link

Workflow run id 9301660792 approved.

@dune-eng
Copy link

Workflow run id 9304240655 approved.

@dune-eng
Copy link

Workflow run id 9304240918 approved.

@dune-eng
Copy link

Workflow run id 9305366837 approved.

@dune-eng
Copy link

Workflow run id 9305366937 approved.

@dune-eng
Copy link

Workflow run id 9306913971 approved.

@dune-eng
Copy link

Workflow run id 9306914108 approved.

@dune-eng
Copy link

Workflow run id 9308932457 approved.

@dune-eng
Copy link

Workflow run id 9308932708 approved.

Copy link
Member

@jeff-dude jeff-dude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pushed a final commit myself, since i forgot to ask in review :) essentially filters the target data to the same timeframe as source data, to make merge faster on incremental runs.

thanks for implementing the feedback!

@jeff-dude jeff-dude added ready-for-merging and removed WIP work in progress in review Assignee is currently reviewing the PR labels May 31, 2024
@alexus-98
Copy link
Contributor Author

thanks for the commit! How long does it usually take to merge the new model into the repo?

@jeff-dude
Copy link
Member

thanks for the commit! How long does it usually take to merge the new model into the repo?

typically same day or next day, but we had some changes in prod end of last week to block us from merging. that isn't typical. i am working through the list of merges today.

@jeff-dude jeff-dude merged commit 07542e0 into duneanalytics:main Jun 3, 2024
1 of 2 checks passed
Copy link

gitpoap-bot bot commented Jun 3, 2024

Congrats, your important contribution to this open-source project has earned you a GitPOAP!

GitPOAP: 2024 Dune Contributor:

GitPOAP: 2024 Dune Contributor GitPOAP Badge

Head to gitpoap.io & connect your GitHub account to mint!

Learn more about GitPOAPs here.

@github-actions github-actions bot locked and limited conversation to collaborators Jun 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants