Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LegiScan API results should be persisted on a 1-hour schedule #28

Open
j1mbl3s opened this issue Feb 1, 2020 · 3 comments
Open

LegiScan API results should be persisted on a 1-hour schedule #28

j1mbl3s opened this issue Feb 1, 2020 · 3 comments
Labels
Feature Improvements or additions to the bot. RFC Request For Comments
Milestone

Comments

@j1mbl3s
Copy link
Contributor

j1mbl3s commented Feb 1, 2020

The API documentation says that, within a 1-hour schedule for search/searchRaw, it will serve unchanged cache data (see page 7).

@j1mbl3s
Copy link
Contributor Author

j1mbl3s commented Feb 1, 2020

Watched bills with changes could be displayed on this schedule while another, less frequent schedule could display the rest of the results, with the scan/query command displaying the results as needed.

@x47188
Copy link
Contributor

x47188 commented Feb 1, 2020

in general we track about 180,000 bills
and have an average 3 hour lag between a state publishing any random
update and the system picking it up.

State schedule vary wildly from near real-time to once a day, and the
Timmy the Intern factor in data entry delay from Clerk staff.

And as it relates to public service Pull API calls there is a 1 hour
cache on any specific set of parameters, though 1-2 checks per day is
sufficient for most common identification/update approaches

– Sean, from Legiscan

@x47188 x47188 added the Feature Improvements or additions to the bot. label Feb 1, 2020
@finnbear
Copy link
Member

finnbear commented Feb 1, 2020

My argument for not locally caching bills is as follows: The scan command, which does the API queries, is run very infrequently (one to two times a day). It is not a common command to use. The other commands (ignore/watchlist/etc.) operate on the database.json/watchlist, which acts like a cache.

@x47188 x47188 added RFC Request For Comments Proposition A proposal labels Feb 2, 2020
@finnbear finnbear added this to the V2 milestone Feb 2, 2020
@x47188 x47188 removed the Proposition A proposal label Feb 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Improvements or additions to the bot. RFC Request For Comments
Projects
None yet
Development

No branches or pull requests

3 participants