Remove the storage mutex

[domenic]

Nobody has implemented the storage mutex. I think we should not pretend this will eventually happen. Instead we should supplement the spec with appropriate and detailed warnings about how cookies and local/session storage are prone to of multi-window corruption. Including some examples would be good.

If there are alternate patterns that we could recommend, that would be even better. But I think the same problem still occurs in e.g. IndexedDB? Or maybe transactions help there.

Tagging in @inexorabletash as a Blink storage person.

Related: #334 for removing the public API navigator.yieldForStorageUpdates().

[inexorabletash]

Indexed DB doesn't have the same problem. Connections and transaction scheduling is well defined (although there's no notification API). I'll take a peek at the storage mutex in HTML and comment if anything in worth salvaging.

[annevk]

I wonder if @rocallahan still has plans for the perfect implementation of all this.

[rocallahan]

No, I guess this ship has sailed.

[annevk]

Sad trombone. @Ms2ger, what about Servo?

[Ms2ger]

I don't know if we've really considered it. CC @jdm.

[jdm]

I have not put any effort into implementing an equivalent solution to the storage mutex in the spec.

[inexorabletash]

Yeah, looks fine to rip it all out, possibly with warnings about raciness sprinkled about. Definitely no impact to Indexed DB or SW's Cache.

The bit about "the length attribute of a Storage object" should probably reinforce the localStorage.length === localStorage.length invariant which should hold even without a mutex i.e. updates to storage from other execution contexts should not be visible to script until the stack unwinds (event loop? microtasks? whatever the current hotness is); we try and ensure that across the platform. I haven't looked at Chromium's code to make sure we actually do that.

(Now would be a good time to solicit feedback on Origin Flags wouldn't it?)

[annevk]

@inexorabletash you cannot hold that invariant without a mutex if you use independent processes for multiple same-origin tabs. (Unless perhaps all the storage is in its own process that uses an event loop of sorts. Perhaps that works?)

[inexorabletash]

Tabs could maintain snapshots of storage state which are updated only on return to the event loop. Again, I'm not sure if Chromium or other browsers actually do that in any/all cases. If we don't want to require such behavior, then we should call out that such an invariant doesn't hold.

[rocallahan]

How would you reconcile conflicting updates, then?

[inexorabletash]

In Chrome, the source of truth for storage is in its own process. Tabs maintain local caches, observe updates that come in between ticks of the event loop (which in turn produce storage events), and send updates (last writer wins).

[rocallahan]

Interesting. Is that for localStorage only, or cookies as well? Maybe we should specify that.

[foolip]

The section "Serialisability of script execution" should be removed together with this.

[foolip]

OK, so in #342 I have blindly removed the storage mutex. What kind of guidance should we add? The affected APIs are localStorage and document.cookie.

[inexorabletash]

@rocallahan - re: cookies - they're different (in chrome)

Getting document.cookie does a synchronous IPC (renderer⇢browser) to get the current state.
Setting document.cookie does an asynchronous IPC to set the new cookie.

There's no implicit synchronization mechanism like only updating state at the event loop, so it is possible for to observe cookie state changing during a script execution block. (I have sample code, but it's pretty obvious.)

[foolip]

@inexorabletash do you think it's likely to be web compatible to have document.cookie behave like localStorage, with updates only when returning to the event loop?

@rocallahan how do these APIs behave in Gecko, especially with multiprocess enabled?

[rocallahan]

I don't know. @annevk might know, or might know who knows.

[jdm]

Cookie setting and getting are synchronous operations from the child process to the parent, and there's no synchronization logic around updating when returning to the event loop. Storage is based around a process-local cache and async updates from the parent, so I would expect that to be stable during each turn of the event loop.

[inexorabletash]

@foolip - My guess is that it's web-compatible, given how subtle it is. It would change the behavior of document.cookie === document.cookie from sometimes failing to always passing, but observing the failure could not be relied on anyway. (I had to have the writing tab hammering the cookie jar with updates and the reading tab busy-waiting between reads.)

That said, I don't think we'd rush to update the code here, although nailing this down before exposing a newfangled cookie API (e.g. to workers) would be a good idea and would merit revisiting the impl.

[foolip]

So for localStorage it sounds like at least Blink and Gecko already have the same model: kept stable during script execution, with changes synced when returning to the event loop. Spec'ing that as "sync things" in step 5 ought to work.

For document.cookie, I think the alternatives would be to make it like localStorage, or to spec that both the getter and setter should be synchronous. Preferences?

@inexorabletash said:

Setting document.cookie does an asynchronous IPC to set the new cookie.

I did a very simple test:

console.log(document.cookie);
document.cookie = 'foo=bar';
console.log(document.cookie);

In Chromium, it looks like the second console.log always has the new cookie value, so the setter can't be entirely async. Do you know if this is guaranteed, or am I just lucky with timing?

[annevk]

Note that document.cookie can also be affected by responses from the network. So I think it would have to be more complicated than synchronous.

[foolip]

Can't that be seen as just another source of unpredictable change, like another tab?

[annevk]

Yeah, I guess that is fairly similar in that respect.

[foolip]

In Chromium, it looks like the second console.log always has the new cookie value, so the setter can't be entirely async.

I guess what's happening is that because the getter is sync, it'll block until the message for the async setter has been processed. Right, @inexorabletash?

[inexorabletash]

@foolip Right - async just means we don't wait for a response. Both sync and async IPC get onto the same queue in the receiver, so the async set is guaranteed processed before the async get. (That's implicit but there's even a comment in the code about that)

[foolip]

OK, that means that the API will look synchronous to scripts. Do you think there is any combination multiple tabs, out-of-process iframes and communication with them where it could be observed that the setter is using async IPC? If not, it's very tempting to spec document.cookie as a synchronous API.

[foolip]

OK, the wording here was a bit tricky, which is why I left it sitting for so long. I think it's now ready for review again, @annevk @domenic please take another look and find all the things that don't make sense!

[foolip]

Note that I didn't try to spec how localStorage works at this point. If anyone thinks that could be made sane, or that it could be interoperably insane, please file a separate issue.

[annevk]

The only thing that's somewhat weird is that now we have a warning on cookies, which looks good, but nothing for storage. I guess you did that because strategies differ in implementations. If that's the case, if you open up a new issue for storage so that defining that one way or another gets done in due course, I would be okay with merging this.

[domenic]

There's a warning for storage analogous to the warning for cookies actually.

[foolip]

I have filed #403 to actually define how localStorage works. I don't plan to work on that anytime soon, but the people in this thread could summarize the model of the browsers they work on, that'd be great.