ePipe-0.14.0

[HOPSWORKS-1387] Upgrade to NDB 8 and Extended hive cleaner

ePipe-0.13.0

• [HOPSWORKS-1768] Restrict the usage of scan operations
• [HOPSWORKS-1791][ePipe] Add support for large XAttrs
• [HOPSWORKS-1800] prov-core xattr - remove scan operations - add cache
• [HOPSWORKS-1776] provenance - link artifacts - feature groups, training datasets, models, experiments
• [HOPSWORKS-1900] epipe mishandling of large xattr when logged xattr size differs from current xattr size

ePipe-0.12.0

• [HOPSWORKS-1609][ePipe] Fix version conflicts in Elasticsearch
• [HOPSWORKS-1618][ePipe] Use boost::log library to implement proper logging
• Add docker deployment
• [HOPSWORKS-1667] featurestore index
• [HOPSWORKS-1712] provenance views - check inodes existance -> no inode -> no view
• Make all warnings into errors
• [HOPSWORKS-1737] Fix search for non demo projects
• [HOPSWORKS-1770] reindex featurestore index

ePipe-0.11.0

• [HOPSWORKS-1578] skip projectId - -1 and deleted project operations
• [HOPSWORKS-1541][ePipe] Skip non searchable fields for schemabased extended metadata
• [HOPSWORKS-1306][ePipe] Add support for HA Elasticsearch
• [HOPSWORKS-1543][ePipe] Retry in case of conflict when adding datasets and projects
• [HOPSWORKS-1308][ePipe] Rewrite and fix the metrics code
• [HOPSWORKS-1516][ePipe] handle the case if XAttrAddAll is received for a deleted inode

ePipe-0.10.0

• [HOPSWORKS-1245][ePipe] Fixes for xattrs handling to be work with the changes introduced in HOPS-1434 and HOPS-1451
• [HOPSWORKS-1246][ePipe] Check for the existence of elastic fields before delete
• [HOPSWORKS-1253][ePipe] Rename operation should read the inodes info from the DB
• [HOPSWORKS-1221] append to ePipe metrics
• [HOPSWORKS-782][ePipe] Synchronize recovery tailer and table tailers to avoid inconsistencies during recovery
• [HOPSWORKS-1226][ePipe] Use Boost:Beast http client library instead of Libcurl
• [HOPSWORKS-1248][ePipe] Adapt the new dataset tracking introduced in HOPS-1483
• [HOPSWORKS-1278][ePipe] The tailer should guarantee the order of epochs during recovery and while waiting for events
• [HOPSWORKS-1279][ePipe] Check for the existence of the xattr parent field before delete
• [HOPSWORKS-1254][ePipe] handle elastic search failures
• [HOPSWORKS-1324][ePipe] Adapt to the new dataset table schema
• [HOPSWORKS-1296] [ePipe] log replay fails on xattr op
• [HOPSWORKS-1459][ePipe] provenance support - file and app
• [HOPSWORKS-1372] Upgrade the ELK stack to 7.2 OSS and use the OpenDistro Security Plugin

ePipe-0.9.0

• [HOPSWORKS-1203] [ePipe] Fixes for JSON handling in the XAttr table
• [HOPSWORKS-781][ePipe] Integrate hive cleaner into ePipe
• [HOPSWORKS-1204][ePipe] Use configuration files instead of command line parameters
• [HOPSWORKS-1219][ePipe] Update CMakeLists to use C++14, Boost 1.70, and CMake 3.5.0
• [HOPSWORKS-1221][ePipe] Add metrics endpoint
• [HOPSWORKS-1233][ePipe] hivecleaner fail during project removal
• [HOPSWORKS-1054] Reimplement the schemaless metadata API to use the extended attributes added in HOPS-1312
• [HOPSWORKS-1246][ePipe] Check for the existence of elastic fields before delete
• [HOPSWORKS-1253][ePipe] Rename operation should read the inodes info from the DB
• [HOPSWORKS-782][ePipe] Synchronize recovery tailer and table tailers to avoid inconsistencies during recovery
• [HOPSWORKS-1226][ePipe] Use Boost:Beast http client library instead of Libcurl
• [HOPSWORKS-1278][ePipe] The tailer should guarantee the order of epochs during recovery and while waiting for events
• [HOPSWORKS-1279][ePipe] Check for the existence of the xattr parent field before delete

ePipe-0.8.0

• [HOPSWORKS-983] [ePipe] Wrong type for long values in query condition handling
• [HOPSWORKS-1056] [ePipe] wrong key type for inode id
• [HOPSWORKS-1058] [ePipe] wrong event ordering during recovery
• [HOPSWORKS-1053] [ePipe] Add support for the extended attributes feature added in HOPS-1312

ePipe-0.7.0

• [HOPSWORKS-831] [ePipe] Use Uint64 as the processing index instead of long
• [HOPSWORKS-780][ePipe] InodeIds are long starting from HopsFS 2.8.2.6
• [HOPSWORKS-859][ePipe] Ensure that the processing queue is not empty before calling pop and top

ePipe-0.6.0

• [HOPSWORKS-560] Refactor and rewrite ePipe to work with Elastisearch 6.2.3
• [HOPSWORKS-452] Add support for rebuilding Elasticsearch index in ePipe
• [HOPS-610] Add rename and setOwner as events in metadata_log
• [HOPSWORKS-646] Fix Inconsistent metadata bug due to parallel data readers
• [HOPSWORKS] [ePipe] Fix a race condition while reading from ndb

ePipe-0.5.0

• [HOPSWORKS-563] added provenance support
• [HOPSWORKS-564] Refactoring ePipe to easily enable or disable different watch units