Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support downloading all data as a zip/targz file #144

Open
shankari opened this issue Oct 18, 2024 · 0 comments
Open

Support downloading all data as a zip/targz file #144

shankari opened this issue Oct 18, 2024 · 0 comments

Comments

@shankari
Copy link
Contributor

shankari commented Oct 18, 2024

Right now, when people want to download data, they can download the data that they view.
So to download any new tranche of data, they would need to select a new time range, wait until all the data loads onto the dashboard, and then download one tab at a time.

This is annoying for people who just want to download the data and work on it locally.

There was a request from a user to basically be able to specify a time range, and then just get a zip file of that time range.
This should be more efficient because we can batch on the server and compress on the server without having to deal with sending raw data over the network for display.

Stretch goal: we could also store the last timestamp for which the user downloaded data (like we do in the pipeline) and just download everything after that until now instead of making them remember that.

Instead of making additional improvements to the data tabs, we may want to pivot to this, and then go back to see what changes are needed to the data pages.

For example, we could limit the range for the data page to only be one week or 10 users or something like that because we have this alternate mechanism for the full download. So then the data pages (along with PygWalker) are just a way to experiment with the filtered, small amounts of data for specific questions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant