Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change recording to create a span of type "record_root" #1703

Open
wants to merge 20 commits into
base: garett/move-span-out
Choose a base branch
from

Conversation

sfc-gh-gtokernliang
Copy link
Contributor

@sfc-gh-gtokernliang sfc-gh-gtokernliang commented Dec 19, 2024

Description

Updated the app context manager to be able to create a span of type "record_root"

Other details good to know for developers

The general approach is the following:

  1. In TruApp's context manager - we create a span of type RECORD_ROOT. This is necessary because:
    a. we're relying on the context manager to do things such as running/deferring the feedback function on record completion
    b. The user might already be actively tracing, so we need a point for which we identify the calls that belong to the record
  2. We introduce a new span type - MAIN, intended for the main entrypoint into the app's calls.
    a. This is because we do not have control over what happens within the recording context, so there might be scenarios where the user can call non-instrumented functions, that we have no way of tracking the input/outputs of.
RECORD_ROOT (This holds the record)
|_ MAIN (this should be the main entrypoint into the app's calls)
   |_ call 1
   |_ call 2
   |_ ...

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to
    not work as expected)
  • New Tests
  • This change includes re-generated golden test results
  • This change requires a documentation update

Important

Add RECORD_ROOT and MAIN span types to TruApp context manager for improved recording and tracing.

  • Behavior:
    • Add RECORD_ROOT span type in TruApp context manager in app.py to manage recording contexts.
    • Introduce MAIN span type in instrument.py for main entry points.
  • Instrumentation:
    • Modify instrument() in instrument.py to handle MAIN span type.
    • Update set_main_span_attributes() in span.py to set attributes for MAIN spans.
  • Testing:
    • Update test_otel_instrument.py to test new span types and attributes.
    • Add new test cases for MAIN and RECORD_ROOT spans.
  • Misc:
    • Add tokens and span_context attributes to App class in app.py for context management.
    • Import changes in app.py to use OTELApp from instrument.py.

This description was created by Ellipsis for f3c9e22. It will automatically update as commits are pushed.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Dec 19, 2024
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Base automatically changed from garett/SNOW-1854278 to main December 23, 2024 19:31
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Dec 23, 2024
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Dec 24, 2024
examples/experimental/otel_exporter.ipynb Show resolved Hide resolved
try:
ret = func(*args, **kwargs)
except Exception as e:
# We want to get into the next clause to allow the users to still add attributes.
# It's on the user to deal with None as a return value.
func_exception = e

if span_type == SpanAttributes.SpanType.MAIN:
all_args = list(args) + list(kwargs.values())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about this for like 15 minutes just now because I don't love this for a few reasons, but I honestly can't think of a good solution to the things that bother me. My main issues are:

  1. If there are kwargs we won't know what keyword they map to.
  2. It's quite likely the first arg is self which I doubt anyone cares to see the string representation of.
  3. I'm not sure if we should just error out if theres multiple inputs (e.g. if all_args is `[self, "hi", "bye"]).

Anyway if you can think of a nicer way I'm all ears but I'm not blocking this approach since it's not like I can see a good approach to replace it with.

tests/unit/test_otel_instrument.py Outdated Show resolved Hide resolved
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jan 3, 2025
@sfc-gh-gtokernliang sfc-gh-gtokernliang changed the base branch from main to garett/move-span-out January 3, 2025 21:22
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Jan 3, 2025

def set_main_span_attributes(
span: Span,
/,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a typo?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted the remaining args to be positional and kwargs for code readability, happy to change this

src/core/trulens/experimental/otel_tracing/core/span.py Outdated Show resolved Hide resolved
src/core/trulens/experimental/otel_tracing/core/span.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants