Add "How Clerk works" and "Client handshake" to initial concepts section

docs/concepts/client-handshake.mdx

@@ -0,0 +1,62 @@
+---
+title: Client handshake
+description: The Client Handshake is a mechanism that is used to resolve a request’s authentication state from “unknown” to definitively signed in or signed out.
+type: conceptual
+---
+
+# The client handshake
+
+Clerk uses a client handshake mechanism to resolve a request’s authentication state from unknown (`handshake`) to `signed-in` or `signed-out`. Clerk’s session management architecture relies on a short-lived session JWT to validate requests, along with a long-lived session that is used to keep the the session JWT fresh by interacting with the Frontend API. The long-lived session token is stored in an HttpOnly cookie associated with the Frontend API domain. If a short-lived session JWT is expired on a request to an application’s backend, the SDK doesn’t know if the session has ended, or if a new short-lived JWT needs to be issued. When an SDK gets into this state, it triggers the handshake.
+
+With the handshake, we can resolve authentication state on the backend and ensure the request is properly handled as signed in or out, instead of being in a potentially “unknown” state. The handshake flow relies on redirects to exchange session information between FAPI and the application, ensuring the resolution of unknown authentication states minimizes performance impact and behaves consistently across different framework and language implementations.
+
+## Handshake flow
+
+The handshake mechanism operates by way of a redirect from the host application to a FAPI endpoint:
+
+1. Request is made to an application using Clerk
+2. Clerk SDK determines the authentication state of the request (`signed-in`, `signed-out`, or `handshake`).
+3. If authentication state is `handshake`, Clerk responds with a 307 redirect to the handshake endpoint: `fapi/v1/client/handshake`
+4. The handshake endpoint gets information about the current session and returns a handshake payload. The encoded handshake payload contains a list of `Set-Cookie` header directives to be passed along with the final response
+    1. If the session is active, a fresh session JWT cookie is returned
+    2. If the session is inactive, the session JWT cookie is wiped and the request will be treated as signed out
+5. The handshake endpoint redirects back to the host application along with the handshake payload, encoded either in the URL (development) or as a cookie (production)
+6. The handshake payload is parsed and `Set-Cookie` headers are set on the response
+7. If an updated session JWT is returned, the JWT is verified
+    1. If successful, the request is treated as signed in
+8. If an updated session JWT is not returned, the request is treated as signed out
+
+## Scenarios that trigger a handshake
+
+<Callout type="warning">
+A handshake is only triggered If a request is determined to be a document request (detected by the presence of the `Sec-Fetch-Dest: document` header, or `Accept: text/html`). Otherwise, the request is treated as `signed-out`.
+</Callout>
+
+1. Development instance and a dev browser is detected in URL **[URL-based session sync]**
+2. Production instance and request is being made to a satellite application **[satellite needs sync]**
+3. Has a session token (`__session`), but no active client is detected (`__client_uat` = 0 or missing) **[session token without client UAT]**
+4. Has active client (`__client_uat` > 0) but no session token **[client UAT without session token]**
+5. Client UAT is more recent than current session token issued at (`__client_uat` > `sessionToken.iat`) **[session token outdated]**
+6. Session token exists but is expired **[session token expired]**
+7. Session token exists but is not active yet **[session token not active yet]**
+
+## Handshake architecture
+
+The handshake flow is composed of two main pieces:
+
+1. The FAPI [`/client/handshake` endpoint](https://github.com/clerk/clerk_go/blob/main/api/fapi/v1/router/router.go#L223)
+2. The [`@clerk/backend` `authenticateRequest()` method](https://github.com/clerk/javascript/blob/main/packages/backend/src/tokens/request.ts#L74)
+
+### `/client/handshake` endpoint
+
+The endpoint should always result in a redirect back to the provided `redirect_url`. Along with the response, a **[handshake payload](#handshake-payload)** is returned. The handshake endpoint also handles multi-domain syncing between a satellite and a primary domain. If possible, the handshake will return a refreshed session token, as well as updated dev browser (`__clerk_db_jwt`) and Client UAT (`__client_uat`) cookies.
+
+### `@clerk/backend` method
+
+The `authenticateRequest()` method from the backend package primarily handles determining the authentication state of a given `Request` object. If the request state is determined to be `handshake`, the method returns returns headers to redirect to the handshake endpoint. These headers must then be set on the response returned by the SDK.
+
+On redirect back to an application from the handshake endpoint, `authenticateRequest()` will run again and parse the handshake payload. Based on the contents of the payload, the request is determined to definitively be `signed-in` or `signed-out` and handled accordingly. The cookie directives from the handshake payload are set as `Set-Cookie` headers on the final response.
+
+### Handshake payload
+
+The handshake payload is a signed JWT that contains an array of `set-cookie` header directives. This allows us to transfer cookies from the FAPI domain to the application’s domain across environments without additional cookie setting logic embedded into our SDKs. In development, the payload is transported in the URL in a `__clerk_handshake` query parameter. In production, the payload is transported in a `__clerk_handshake` cookie. On returning from the handshake endpoint, the handshake payload is parsed and the cookie directives are set on the final response.

docs/concepts/how-clerk-works.mdx

@@ -0,0 +1,106 @@
+---
+title: How Clerk works
+description: 
+type: conceptual
+---
+
+# How Clerk works
+
+## Development and production instances
+
+Clerk provices both development and production environments for your app. Production instances are used for production, pre-production, or staging environments with fixed domains. Production instances offer the strongest security and performance guarantees. Development instances contain certain trade-offs meant to support ease of active development. 
+
+<Callout type="warning">
+Never use a development instance in a production environment. Development instances don’t have strong security guarantees.
+</Callout>
+
+## Main objects
+
+Clerk uses five main objects:
+
+### [Client](/docs/references/javascript/client#client)
+
+A client represents the current device or software accessing an application such as your web browser, native application for Android or iOS, Chrome Extension, or Electron app.
+
+### [Session](/docs/references/javascript/session)
+
+A session is a secure representation of the authentication state of the current user. Each client can hold multiple sessions on the same device. This is identical to how Gmail works in a browser.
+
+### [User](/docs/users/overview)
+
+A user represents the current user of the session. The object holds all the basic user information e.g. name, email addresses, phone numbers, etc… including their public, private, and unsafe metadata.
+
+### [Organization](/docs/organizations/roles-permissions)
+
+An organization represents the current organization of the session. Users can belong to many organizations. One of them will be the current organization of the session. 
+
+### [Roles](/docs/organizations/roles-permissions)
+
+A user belongs to an organization with a role that defines their permissions. Currently, Clerk supports two roles, `org:admin` and `org:member`.
+
+## Clerk Frontend API (FAPI)
+
+Every Clerk development and production instance has a dedicated [Frontend API](https://clerk.com/docs/reference/frontend-api). This is the authentication, session, user & organization management API you or your users will interact with within the web browser, native application, Chrome extension, or Electron app.
+
+Clerk assigns a unique random API URL such as `https://delicate-wahoo-73.accounts.dev/sign-in` to each development instance. For production instances, the Frontend API lives at `https://clerk.[your-domain.com]`.
+
+### How it works
+
+Clerk's Frontend API is responsible for maintaining the authentication state of the current client (browser or native application). Based on the authentication's state (`signed-in`, `signed-out`, `handshake`) it mints a short-lived session [JSON Web Token (JWT)](/docs/backend-requests/making/jwt-templates) for the current user and session to be used by the host application that is secured by Clerk. The session JWT is stored in a cookie or can be retrieved at any time via the [`getToken()`](/docs/references/javascript/session#get-token) method.
+
+### Clerk cookies
+
+Clerk’s Frontend API uses two cookies for session management in production instances:
+
+* `__client` cookie: a long-lived session that is used to keep the the session JWT fresh by interacting with the Frontend API.
+* `__session` cookie: a short-lived session JWT to validate requests on your application or your API. 
+
+#### The `__client` cookie
+
+This cookie is set on Clerk Frontend API (for example `https://clerk.<example.com>`) for each instance. It’s HTTP-only, first-party, and secure. It contains a long-lived client JWT that lasts 7 days by default. The duration is configurable in the Clerk Dashboard's  [Sessions page](https://dashboard.clerk.com/last-active?path=sessions). The JWT identifies the current client (browser, native application, or chrome extension) and sets the current active sessions.
+
+#### The `__session` cookie
+
+This cookie is set on the host application that is secured by Clerk (for example `https://dashboard.<example.com>`). It’s a JS cookie, it’s secure, and contains a short-lived session JWT that lasts 60 seconds. This JWT contains the current session, user, and organization identifiers and must be sent to the application API to authenticate API requests. In SSR, the `__session` cookie travels automatically to the server. ClerkJS ensures that the `__session` cookie is automatically refreshed in Client Side Rendering (CSR) and Server Side Rendering (SSR), ensuring a fresh session is always available.
+
+However, if a `__session` cookie is expired on a request to an application’s backend, the SDK doesn’t know if the session has ended, or if a new short-lived JWT needs to be issued. When an SDK gets into this state, it triggers Clerk's client handshake.
+
+### The client handshake
+
+Clerk uses a client handshake mechanism to resolve a request’s authentication state from unknown (`handshake`) to `signed-in` or `signed-out`. 
+
+With this handshake, Clerk can resolve authentication state on the backend and ensure the request is properly handled as signed in or out, instead of being in a potentially unknown state. The handshake flow relies on redirects to exchange session information between FAPI and the application, ensuring the resolution of unknown authentication states minimizes performance impact and behaves consistently across different framework and language implementations.
+
+Learn more about [how Clerk's client handshake works in this deep dive](/docs/concepts/client-handshake).
+
+### Client piggybacking
+
+Every Frontend API request except from `/v1/client/sessions/:id/tokens` that refreshes the Clerk session tokens, returns a response payload that contains two top level keys
+
+```json
+{
+   client: ...,
+   response: ...
+}
+```
+
+The `response` key contains the resource or resources of each endpoint. The `client` key contains the client piggybacking payload. The client object identifies the current device and contains all sessions, users and organization data for the current device (see [Main Objects](#main-objects) section). 
+
+This is effectively the necessary frontend state for ClerkJS and powers the JS SDKs, the React hooks and the Components. It is a best-effort way for the backend to update the frontend state on every Frontend API request. 
+
+It's not recommended to use Client piggybacking because of the following issues:
+
+1. It’s a very large chunk of a tree-like state and doesn’t change often.
+1. It’s hard to compute on the backend as it requires a lot of data from the database. The computation takes place on every request adding a significant overhead.
+1. It’s best effort.
+1. It’s not a two-way communication channel.
+
+### Clerk Frontend API for non-standard web browsers
+
+Clerk offers a seamless authentication solution for non-standard web browser environments such as React Native applications, Chrome Extensions, and hybrid apps in platforms like Capacitor.js or Electron.
+
+These platforms treat cookies differently than web browsers. As a result, ClerkJS should use the HTTP Authorization header instead of the `__client` cookie for secure communication with the Clerk Frontend API. To achieve session persistence, the `__client` JWT should be stored in a secure storage provided by each platform and then it should be injected into every Clerk Frontend API request. 
+
+## Clerk Backend API (BAPI)
+
+Clerk's [Backend API](https://clerk.com/docs/reference/backend-api) is a restful CRUD API for the server side. It allows a sudo-level management of all Clerk objects.

docs/manifest.json

@@ -23,6 +23,17 @@
       ["Fastify", "/quickstarts/fastify"]
     ]
   ],
+  [
+    {
+      "title": "Concepts",
+      "icon": "lightning-bolt",
+      "root": "concepts"
+    },
+    [
+      ["How Clerk Works", "/concepts/how-clerk-works"],
+      ["Client Handshake", "/concepts/client-handshake"]
+    ]
+  ],
   [
     {
       "title": "Guides",