TruLens Eval v0.10.0

Library containing evaluations of LLM Applications

Changelog:

• Allow connecting to remote database via SQLAlchemy

Bugfixes:

• Patch instrumentation losing track of instrumented objects (Fixes #373)

TruLens Eval v0.9.0

Library containing evaluations of LLM Applications

Changelog

• Allow custom feedback function naming
• Allow multi output from feedback functions via key, value output
• Display method name alongside class name in timeline view
• Support for calls to langchain evaluation criteria
• Allow instrumentation tracking of user defined classes for LLM apps

Examples

• Added Langchain Agents example

Bug Fixes

• Fix For get_records_and_feedback when supplying app_ids
• Silence key not supplied errors on all API calls
• Fix check on AzureOpenAI provider for groundedness feedback function

TruLens Eval v0.8.0

Library containing evaluations of LLM Applications

Changelog

• Support for asynch calls for both langchain (acall) and llama-index (aquery)
• Support for streaming and chat for llama-index (chat, achat, stream_chat, astream_chat)
• Support for user subclassed components for both langchain and llama index
• Support for specifying a db file via Tru(database_file=“dbfile”)

Examples

• Added a FAISS example
• Examples added for asynch
• Fix reference to correct literature - Alice in Wonderland in llama-index subquestion example
• Add groundedness to pinecone and agents examples
• Add colab links to examples

Bug Fixes

• Fix AzureOpenAI to take deployment_id
• Fix code bugs

TruLens Eval v0.7.0

Library containing evaluations of LLM Applications

Changeling

• Added Groundedness Feedback functions to verify supporting evidence using Huggingface NLI and OpenAI LLMs
• Updated UI Timeline view to include application component input and output details on click
• Updated UI leaderboard to show application metadata

Documentation

• Added Prompt/Response and Question/Statement Performance to API documentation for visibility into the feedback function working on real data
  • https://www.trulens.org/trulens_eval/pr_relevance_smoke_tests/
  • https://www.trulens.org/trulens_eval/qs_relevance_smoke_tests/
• Added TruBasicApp to API documentation

Examples

• Updated llama-index agent example with more evaluations

Bug Fixes

• Removed key requirements from some UI components introduced in 0.6.0

TruLens Eval v0.6.0

Library containing evaluations of LLM Applications

Changelog

• Added a feedback function (and notebook example) that checks against provided ground truths
• Added a visibility framework into feedback function operations (what data is being used by the function, etc)
  • More feedback functions will add useful information in the future
• Improved QS Relevance function and added human validated quality checks which will be added to documentation in the future
• Added warning levels (yellow) for feedback functions

Examples

• Added example interfacing with Pinecone DB
• Added example using llama-index agents

Dependency Upgrades

• Upgraded llama-index to version 0.7+

Bug Fixes

• Add error handling around non-importable functions for deferred evaluation

TruLens Explain

Library containing attribution and interpretation methods for deep nets

TruLens Eval

Library containing evaluations of LLM Applications

Changelog

• Upgraded the Record Viewer to include Framework call stack, call timing, and merged the call parameters into the call definition dropdown.
• Added a simple CLI on installation to start the dashboard with a single trulens-eval call
• Added support for non-framework LLMs such as direct OpenAI LLM API calls or any generic "text to text" applications
• Added new examples including the evaluation of different configurations with Pinecone vector database
• Added support for Colab notebooks and added linked examples to the README
• Added ability to add tags to Apps
• Added usage documentation for Feedback Function input selection
• Bugfixes for better error handling from Feedback Functions

TruLens Eval

Library containing evaluations of LLM Applications

Changelog

• Backwards compatibility for previous versions
• A large set of new examples in trulens/trulens_eval/examples
• AzureOpenAI Support
• Various Bug fixes for Dashboard, DB serialization, Llama index subqueries, Huggingface API calls

TruLens

Library containing attribution and interpretation methods for deep nets

TruLens

Library containing attribution and interpretation methods for deep nets