daa.analytics v0.7.0

New Features

• Updated the code to accommodate the new disaggregations of 65+ for specific indicators.
• We included a timestamp functionality to allow us to keep track on the last time S3 bucket was updated.
• Through a DATIM API, we were able to get more facility level data for analysis, that is data for DSD and TA.
• Implemented a new formula for calculating the global concordance, this will enable countries compare how they're performing against the global results for a specific indicator and year.
• Included a descriptive information for country users under the "Reporting Rates" tab for countries that do not have either PEPFAR or MOH data for the current fiscal year.
• On the Reporting Rates tab, we added raw numbers on top of each bar chart to show actual figures of each indicator.

Enhancements

• We changed the R code settings on the Rstudio platform to allow more people to log in simultaneously and pull data, this enhancement increased the number of active sessions from 3 to 5.

Bug Fixes

• On the raw data download, we removed duplicate columns "emr_present" and maintained one of them which we then verified that the information provided for a facility if it has EMR in that site, correlates with what is on DATIM, we used a DATIM api to pull all facilities and compared to all facilities we had on the Explorer App for all PEPFAR supported countries and confirmed that the presence of EMR is similar on both datasets for all facilities.
• Fixed an issue with the dates under subtitles whereby when a user changes a filter date, the subtitle with the date was not changing but rather stayed static, now it is responsive of what the user selects on the filter side. This mainly affected the Reporting Rates tab and the Concordance Graph.

Performance Improvement

• Improved the performance of the overall application by optimizing specific functions responsible for taking up a huge load time. The function "get_daa_data" pulls millions of rows, we removed unnecessary function loops and used inbuilt recommended R functions to make it more efficient. This reduced our time to pull all data for all countries from 24 minutes to 20 minutes.

Other Changes

• We adjusted the size of the scatterplot in the Site Scatterplot tab to make it fill up more of the page and not be bunched up at the top.
• We adjusted the size of the concordance graph in the Concordance Graph tab to make it more visible to users.
• We removed the raw numbers of tx_pvls_n and tx_pvls_d from the raw data download by removing the code handling those figures on S3 bucket. Since we have them as indicators now, we do not need them as individual columns on the raw data files.
• We updated the global summary file by including a column which shows the global concordance percentage of all countries for each year and each indicator. We also included two new suggested columns for further analysis and re-formatted the naming of the columns.
• On the site data tab, we removed a column called Weighted Concordance(OU Level) from the table and we also made the column called Difference to be absolute values and not have negatives.

Known Issues

• There is still room for improvement on the load time the system takes to load all data. We can make further improvements on caching especially for data coming from the S3 bucket specifically the dataset handling VLS and EMR called pvls_emr_raw.
• We can also see way to further optimize the functions that fetch data from DATIM.

Contributors

@Christian-Onyango @bangadennis @vshioshvili @jknuep

Feedback

We welcome your feedback on any suggestions for improvements or issues encountered.

[Christian Onyango - Software Engineer, ICF.]

daa.analytics v0.6.2

Bug fixes

• Fixed bug on replacing NA values with FALSE on the file adorn-pvls-emr.
• Fixed bug on the combined-data file pulling with NULL on the OU column.
• Merged the global summary file last two columns to have the new mapping format and the old mapping format into one

Breaking changes

*On adorn-weights file, we used native R pipe for function chaining.
*Included the column attributeOptionComboIdScheme in the file get-daa-data to fix 2.38 API updates.

New features

*We changed the import-history file to accomodate data format for 2021 onwards. This included logic changes in the code and addition of new columns.
*We separated the indicator TX_PVLS to have a numerator and a denominator value as requested, this ensured the total number of indicators we have amounted to 8.

Minor improvements and fixes

*Updates R from version 4.1.1 to version 4.2.1.
*Updates versions of several required and suggested packages.

daa.analytics v0.4.0

Bug fixes

• Fixes minor bug with how adorn_weights and combine_data handle
  Organization Units with duplicative UIDs.

Breaking changes

• Deprecates get_coc_metadata, get_de_metadata, get_ou_metadata,
  get_pe_metadata, and get_pvls_emr_data for a single function called
  get_s3_data which takes in an argument named dataset_name to indicate
  which file to update.
• Renames get_data_availability to get_import_history.
• Renames weighting_levels to adorn_weights.

New features

• get_s3_data and fetch_s3_files now both allow the user to only update
  files if the data has changed on S3 since the last time a file was grabbed.

Minor improvements and fixes

• Updates R from version 3.6.3 to version 4.1.1
• Updates versions of several required and suggested packages
• New data-raw files added for updating data for a single OU and
  for saving CSVs with country data.

daa.analytics v0.3.0

Breaking changes

• adorn_daa_data now no longer exports columns county_of_matched_sites,
  pepfar_sum_at_matched_sites, weighting, weighted_discordance,
  or weighted_concordance.
  • count_of_matched_sites and pepfar_sum_at_matched_sites will now no
    longer be supported.
  • weighting, weighted_discordance, and weighted_concordance will be
    replaced by weights and metrics calculated at each level of the organisation
    hierarchy going forward and will be calculated using the weighting_levels
    function.

Experimental features

• weighting_levels is a new function that calculates weightings as well as
  concordance and discordance metrics for DAA indicators at all levels of the
  organisation hierarchy.

Minor improvements and fixes

• Adds UIDs for each organisation hierarchy level to ou_hierarchy dataset
• Documentation updates
• Adds NEWS.md file

daa.analytics v0.2.0

Upgrades to include dataelements for the FY2021 activity, as well as minor code and documentation cleanup.

daa.analytics v0.1.6

Corrects error with processing of 2021 DAA data availability data from GeoAlign due to inconsistent naming conventions.

daa.analytics v0.1.5

Increases the number of times API call attempts are made to DATIM to the analytics and attributes endpoints to increase reliability of functions when calls fail on the first attempt.

daa.analytics v0.1.4

• Refactors adorn_pvls_emr function to put all EMR data into a single column mapped to both indicator and facility instead of just facility.

daa.analytics v0.1.3

Updates vroom package to 1.5.5 to resolve errors.

daa.analytics v0.1.2

Fixes error from dplyr version 1.0.7 by reverting to version 1.0.6.