What is an Agent?

[annevk]

I'm trying to figure out how to embed the Agent concept into the HTML Standard. It's clear that a worker, shared worker, service worker, or worklet all represent a single agent.

It's not clear for Window/document. @lars-t-hansen suggests it's meant to match the concept of document, however that would mean we'd end up with two Agents and one global for initial about:blank navigation.

Previously it was suggested it's meant to match the event loop concept, but that's also not true as that much more closely matches what you call "execution thread" in scope (not necessarily in function, although I think it's close).

So it would be good to hear about some invariants or ways to test what is actually implemented and maybe get the specification clarified what it actually means since it's rather vague.

SharedArrayBuffer integration issue for the HTML Standard: whatwg/html#2260 (also whatwg/html#2361 and whatwg/html#2518).

[lars-t-hansen]

Well, "document". You can't have an agent (in any meaningful sense) without a global object / Realm since every execution record requires a realm and an agent only works in terms of processing execution records. So when I say document I probably really mean the instantiated ES environment you get when you have a document. I know about:blank is a thing but it seems like a special case that could be adapted to fit agents somehow.

[annevk]

The instantiated ES environment would match up with the Window object I think (which ends up being reused for initial about:blank navigation whereas the document gets replaced). That would mean Agents are 1:1 with what HTML calls settings objects, or settings object's global objects.

See also https://html.spec.whatwg.org/#realms-settings-objects-global-objects (if the Realms API becomes a thing that'll need to change obviously and someone should work through the implications of that changing).

[domenic]

Hmm. I was pretty sure that agents were supposed to be event loops. For example there are many realms within a single agent.

[domenic]

For example I thought that two same-origin iframes were in the same agent. E.g. they share a list of promise jobs (which are agent-scoped). In general I thought agent was supposed to represent any code which can synchronously access other code's memory from JavaScript today. And agent cluster was supposed to represent any code which can access other code's memory using SAB, which has more capabilities than JavaScript today.

@lars-t-hansen is that not the case?

[domenic]

If agents are 1:1 with realms we should delete the agent concept from ES262 and just use realms everywhere.

[erights]

Maybe I am missing some context, but Agent and Realm are quite clearly totally distinct concepts. Realm is about a set of global objects. Agent is about concurrency. Job queues are per-Agent rather than per-Realm. In JavaScript, any given Realm must be within one Agent, but an Agent can contain any number of Realms. I don't see how these can be confused.

In the browser, Origin cross-cuts Agent. Any Realm must also be within one Origin and an Origin can contain any number of Realms. But neither Origins nor Agents contain each other. An Agent can contain Realms from different Origins and an Origin can contain Realms running in different Agents.

At the last TC39 meeting, where we discussed reifying Realms with a Realm API, the question came up about reifying Agents and Job queues into an Agent API. This should be done eventually, but will be almost completely distinct from the effort to create a Realm API.

[annevk]

@erights that also matches my understanding, with the further clarification that Agent encompasses all Realms that can possibly execute each other's functions within the same event loop (all "similar-origin" Window objects), but not more. So a.yes.no and b.yes.no are same Agent and a.yes.no a.maybe.no are not. (That distinction matters if we accurately want to define who can have shared memory and who cannot.)

However, when I talked with @lars-t-hansen about it on IRC today he said that Agent roughly corresponded to "document".

[allenwb]

Agents can't be 1:1 with realms because realms to do have any of the resource listed in the first paragraph of the Agent Definition privately associated with them.

It appears to be that while the definition of Agent in 8.7 is probably adequate to described the semantics of inter-Agent sharing shared buffers, it is missing a few things to complete the ECMAScript language semantics.

The following needed to be added as constituent elements of an ECMAScript Agent:

• A set of Realms. The realms in the set are private to the Agent and may not be accessible to any other agent.
• A heap of SharedArrayBuffer objects that whose access may be shared with other Agents
• A set of well-known symbol values
• A GlobalSymbolRegistry

An event loop is not included because it is not (yet?) an ECAMScript language concept. But in an implementation it manifests via the per agent job queue and job schedule.

[domenic]

To be a bit clearer, this is an issue cross-cutting ES and HTML, so that is why we are talking about things like origins and event loops that are outside the scope of ES.

It sounded like from @lars-t-hansen's description that in implementations agents and realms are 1:1 (and would be forever). In that case I was suggesting we merge the two spec concepts. We could also keep them separate (just like we keep global object and Realm separate, instead of e.g. using lots of slots on the global object) but that would just be a spec formalism, and IMO a dubious one.

But it sounds like that is probably a moot discussion since at least @erights, @allenwb, and myself believe that @lars-t-hansen's description of agents was mistaken. In particular @allenwb's

The realms in the set are private to the Agent and may not be accessible to any other agent.

definitely makes it impossible for each agent to contain only one realm (in a browser) since many realms can access each other synchronously (in a browser).

What remains is the confusion that the creator of agents (@lars-t-hansen) has a different conception of them than us three, and believes they are 1:1 with realms. It'd be ideal to sort that out.

[jfbastien]

If WebAssembly gets its own threads (WorkerLite™), would these be Agents? Is that out of scope for TC39? I think we intend to re-use the wording and want it to inter-operate, e.g. SharedArrayBuffer can be shared between WebAssembly and JavaScript.

[syg]

@domenic @annevk It doesn't make sense for Agents to be 1:1 with Realms, neither in spec nor in implementation. If they were 1:1 in spec, it would mean currently cross-Realm execution with SAB would be undefined. If they were 1:1 in implementation, that'd mean each global had its own event loop. So all that is just a long-winded version of "I agree with what @erights said". I'd interpret Lars's "document" and ES environment comment as saying that an ES environment must be present for the concept of Agent to be sensible, not anything about its bijection with document.

@jfbastien A by-the-letter reading of 8.7 means wasm threads aren't Agents because they don't have any ES-isms like Realms, ES execution contexts, etc. For the sake of interop, I think we'd need to factor out a notion of Agent that doesn't include ES things but does include stuff in that table, the various boolean flags and the candidate execution for the memory model.

[binji]

@syg I started to work on that for the WebAssembly atomics proposal here WebAssembly/design#1019, but it's pretty informal.

[annevk]

Yeah, I later recalled the whole debate about naming agents continents so @lars-t-hansen must have meant some other kind of document.

I like @allenwb's suggestions. I think we could go even further and have CreateAgentCluster and CreateAgent abstract operations that detail the arguments a host is expected to provide (such as [[CanBlock]] for agents), though it might be quite some time until we have refactored the HTML Standard enough to be able to use these properly (our lifetime story is not great).

(WebAssembly integration can be discussed at #887.)

[lars-t-hansen]

What remains is the confusion that the creator of agents (@lars-t-hansen) has a different conception of them than us three, and believes they are 1:1 with realms. It'd be ideal to sort that out.

I think when Anne says "document" he means something much more specific than I mean, and there is every reason to be skeptical when I casually use something that sounds like they may be HTML terms.

The final intent of the interaction of SAB and HTML was recorded here: http://tc39.github.io/ecmascript_sharedmem/shmem.html#WebBrowserEmbedding. Per discussions with Domenic we settled on a mapping s.t. in an HTML embedding, an Agent is realized by having the event loop provide the the executing thread.[*] This was to allow for situations where code in one realm (the tab) reaches into another realm (same-origin iframe, opened window). Accounting for the HTML embedding and browser semantics influenced the SAB spec a little bit, in that (a) the executing thread can be shared (if the tab reaches into an iframe they cannot be running concurrently) and (b) shared threads cannot block (or programs would deadlock quickly). Other than that, I tried to keep the Agent spec as open as possible.

[*] I dislike saying that an Agent "is" an event loop because an Agent is already defined in the ES spec and it is its own thing. The embedding has to provide the machinery the spec requires to drive evaluation however. Those "holes in the spec" are not well spelled out, see also @allenwb's list. (It does not help the matter that the link quoted above states that an agent "is" an event loop, nor does my informal use of HTML terms.)

[annevk]

Glad it was just confusion. (By document I do indeed mean an object that is hold by the Window object (the global object in use for non-worker scenarios) and typically has a 1:1 relationship with that object, but not always.)

I think as a minimum we should give Agent a [[Realms]] slot and then we can say which realms belong to a single agent and what that agent's [[CanBlock]] should be. That also avoids comparisons with the event loop and changing the definition of the event loop (as it is currently defined in HTML it can be used across agent clusters and that is also how user agents implement it, definitely for cross-origin iframes, though this may change in the future).

[domenic]

So shall we leave this open to track @allenwb's #882 (comment), or should we maybe close an open a new issue to track that specific work item?

[annevk]

Now done, see #1357.