Skip to content
This repository has been archived by the owner on Nov 7, 2020. It is now read-only.

Test inconsistency #267

Open
yourcelf opened this issue Mar 5, 2014 · 3 comments
Open

Test inconsistency #267

yourcelf opened this issue Mar 5, 2014 · 3 comments

Comments

@yourcelf
Copy link
Collaborator

yourcelf commented Mar 5, 2014

test/test.session-joining.selenium.js seems to fail fairly often, but inconsistently. It reliably passes when executed individually, but occasionally when executed with the whole test suite, it fails about half the time like these:

https://travis-ci.org/drewww/unhangout/builds/20160219
https://travis-ci.org/drewww/unhangout/builds/20159670
https://travis-ci.org/drewww/unhangout/builds/20146641
https://travis-ci.org/drewww/unhangout/builds/20095832
https://travis-ci.org/drewww/unhangout/builds/20095636

This seems to be either some sort of global state issue, some sort of timing issue (which only manifests in the whole-test-suite context), or some sort of socket/open files type issue. Not sure what's going on.

@yourcelf
Copy link
Collaborator Author

yourcelf commented Mar 7, 2014

I've worked around this problem by splitting the test suite in two during runs (see 9400319 ). Leaving this open as I'd prefer to fix it without resorting to hacks like that.

yourcelf added a commit that referenced this issue Mar 27, 2014
yourcelf added a commit that referenced this issue Apr 1, 2014
Specifically: disallow setting connectedParticipants from users who have
the wrong hangout-url.  May be adequate to fix #280.

Also, do some more not-entirely-fruitful work on addressing test
inconsistency (#267).
yourcelf added a commit that referenced this issue Apr 4, 2014
yourcelf added a commit that referenced this issue Apr 9, 2014
yourcelf added a commit that referenced this issue Apr 9, 2014
yourcelf added a commit that referenced this issue Apr 11, 2014
yourcelf added a commit that referenced this issue Apr 15, 2014
yourcelf added a commit that referenced this issue Apr 17, 2014
@yourcelf
Copy link
Collaborator Author

yourcelf commented May 5, 2014

After a long chain of fixes, I think most of the myriad timing issues have been identified and fixed. At this point, tests run consistently enough that I think we can regard any new inconsistencies as separate issues, and this catch-all can be retired.

@yourcelf yourcelf closed this as completed May 5, 2014
@yourcelf
Copy link
Collaborator Author

yourcelf commented Sep 4, 2014

Reopening due to failures in #340.

@yourcelf yourcelf reopened this Sep 4, 2014
yourcelf added a commit that referenced this issue Sep 13, 2014
yourcelf added a commit that referenced this issue Sep 27, 2014
throttleSync previously used a key of "roomId" + "type" to determine
whether a message was the same type of message, and could therefore be
dropped.  However, since the message transport refactor last April, all
the "types" are simply "state", with paths, operators, and other crucial
details in the args.  Change this method to use a caller-defined key
instead.

Sessions were not being persisted server-side on changes to
connectedParticipants due to a missing return value.

Superagent requests seems to interact poorly with sinon fake timers --
in particular, clock restoration in the body of a request.end callback
seems to not work as expected, leading to at least some of the
inconsistency contributing to #267. Symptom is that "throttleSync" was
triggering functions, but never executing (the setTimeout was never
executing).  Fix this by re-establishing global timers manually.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant