Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HEAP] update browser destination code to load heap js v5 script #2665

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

cpsoinos
Copy link

@cpsoinos cpsoinos commented Jan 2, 2025

Summary

Update the Heap browser destination to use v5 of Heap's javascript snippet.

The Problem

Customers are running into issues with TikTok. We've seen that TT browser web views will strip duplicate key/value pairs on GET requests as an "optimization". Our request schema uses an arbitrary key for custom properties. For example, we'll have the query params: k=name1&k=false&k=name2&k=false&k=name3&k=true, but because of the way TT "optimizes", it will intercept the request before firing off and remove any duplicate query param values, leading to the request being fired having: k=name1&k=false&k=name2&k=name3&k=true, which ends up causing mismatched k/v pairing on the custom properties when we parse the request on the server-side.

The Solution

By upgrading to v5, we replace GET requests with POST so this fixes the issue.

Visuals

Loading the Classic SDK

  • the script heap-${appid}.js is loaded
  • track calls result in a GET request to <host>/h?<params>
  • identify calls result in a GET request to <host>/api/identify_v3?<params>
  • track and identify calls may also cause a GET request to <host>/api/add_user_properties_v3?<params>
Screenshot 2025-01-03 at 1 22 49 PM Screenshot 2025-01-03 at 1 29 33 PM Screenshot 2025-01-03 at 1 29 37 PM

Loading the Latest SDK

  • the script /config/${appid}/heap_config.js is loaded
  • the script /v5/heapjs-static/5.2.6/core/heap.js is subsequently loaded, using the version retrieved within heap_config.js
  • track calls result in a POST request to /api/capture/v2/track with attributes in the payload
  • identify calls result in a POST request to /api/capture/v2/identify with attributes in the payload
  • track and identify calls may also cause a POST request to /api/capture/v2/add_user_properties with additional attributes
Screenshot 2025-01-03 at 1 46 39 PM Screenshot 2025-01-03 at 1 43 31 PM Screenshot 2025-01-03 at 1 52 33 PM

Testing

Include any additional information about the testing you have completed to
ensure your changes behave as expected. For a speedy review, please check
any of the tasks you completed below during your testing.

  • Added unit tests for new functionality
  • Tested end-to-end using the local server
  • [If destination is already live] Tested for backward compatibility of destination. Note: New required fields are a breaking change.
  • [Segmenters] Tested in the staging environment
  • [Segmenters] [If applicable for this change] Tested for regression with Hadron.

@@ -58,9 +60,16 @@ export const destination: BrowserDestinationDefinition<Settings, HeapApi> = {
required: false
},
trackingServer: {
label: 'Tracking Server',
label: 'Tracking Server (deprecated)',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @cpsoinos looks like you have labeled this as deprecated. Is it still referenced anywhere in code though? i.e is it actually used at all?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @joe-ayoub-segment, I updated my changes so that this setting is still used, though it is still deprecated. Customers that wish to self-host will have set both trackingServer and hostname. When both of these settings are set, Segment will load the Classic SDK (retaining current functionality).

type: 'string',
required: false
},
ingestServer: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: adding a new 'optional' setting is OK.

@joe-ayoub-segment
Copy link
Contributor

hi @cpsoinos thanks for the PR.

Couple questions:

  1. Are there customers who are already using this Integration? If so, how can we be sure that this change doesn't break the Integration for them?
  2. Any chance you could add some proof of end to end testing with the new code please?

@joe-ayoub-segment
Copy link
Contributor

Looks like some tests failing in CI also.
image

@cpsoinos
Copy link
Author

cpsoinos commented Jan 3, 2025

hi @cpsoinos thanks for the PR.

Couple questions:

  1. Are there customers who are already using this Integration? If so, how can we be sure that this change doesn't break the Integration for them?
  2. Any chance you could add some proof of end to end testing with the new code please?

Hi @joe-ayoub-segment, thanks for the review.

To address your questions:

  1. Yes, this integration is currently being used. It looks like our destination action isn't available yet? I'll look more into this.
    I've included some logic to fall back to loading the Classic SDK, which is what the currently published version of this Integration uses, for users that are self-hosting the Heap script—though it's unlikely that a customer would opt to self-host but then use a different third party library to load it 🤔 . (I have a query out to other members of my team to see if we can find how many, if any, customers do this.)
  2. I've added screenshots to the PR description showing event calls made from Segment's Actions Tester UI, the resulting network requests, and those events propagated into Heap's Live Data Feed in production.

FYI, I will be OOO all of next week, so if additional changes are required I will address them when I return the week of Jan 13. Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants