Module Context

The backend registry and raw backend entry points. This is the supported path for consumers that genuinely need backend-level operations, such as Parallel and allocator/backend tests. Backend-independent interfaces should name their types through Ir instead.

The cc backend's own module, re-exported because the library's interface module is this one and a test outside arrayjit can otherwise not name it. What needs naming is Cc_backend.compiler_command: the generated-kernel census (test/operations/cc_march_census.ml, gh-ocannl-650) compiles emitted sources under foreign -march flags, and it must do so with the SAME toolchain that builds them for real -- a census against a different compiler would describe guarded arms nothing here selects. The interface is cc_backend.mli's, so this exposes no more than that file already publishes.

The cc backend's builtins table, re-exported for the same reason as Cc_backend above: it is the enumeration of the whole-vector storage bridges C_syntax.vec_bridge renders calls to (OCANNL_VEC_WIDEN_BFLOAT16 and its siblings), and the generated-kernel census (test/operations/cc_march_census.ml, gh-ocannl-752) claims that every one of them reaches an emitted kernel. Deriving that set from the table is what makes the claim fail when a bridge is added and the fixture does not grow to cover it; a list of names in the test would instead be a copy asserting that it still says what it says. Each entry is a (key, definition, dependency keys) triple.

What this host offers the backends, as their own probes found it, without opening any device (gh-ocannl-1038): Cc_backend.host_capabilities (CPU, ISA, vector width, fp16 support, the parallel-Grid runtime, the compiler and its flags), then the host-side probes of each GPU backend whose library is linked -- the CUDA toolkit include tree, the HIP SDK and rocWMMA header trees and the hiprtc version, the Metal math API the runtime accepts. Key/value pairs, in a fixed order.

tools/host_caps.exe prints it as the host line of every CI build job, and test/config/dune records it beside the ambient environment in ocannl_ambient, which every test's configuration carries: a capability that moves -- a header tree appearing at a path already searched, an OpenMP runtime installed, a toolkit symlink repointed -- reruns every test, rather than leaving dune to serve a verdict computed on the host as it was. Device properties (compute capability, device limits, the GPU's architecture) are not here: reading them opens the device, which a probe run on every dune invocation must not do.

Execution context managing device, compilation, and buffers

The routine's executable schedule (its backend task). Abstract on purpose: dispatch goes through run, which validates the lineage and launch bindings before running and records the execution in the ledger afterwards — a directly runnable task would be a one-field-access bypass of those checks.

A compiled computational routine ready for execution. The record is private: only compile constructs routines — the ledger's identity and dependency tracking rely on that — while every field stays readable, so tests can assert on the inputs/outputs the link actually computed instead of re-deriving them (gh-ocannl-590).

Create a CUDA context on the backend's device ordinal (default 0).

Create an AMD HIP (ROCm) context on the backend's device ordinal (default 0).

Create a Metal context on the backend's device ordinal (default 0).

Create a CPU context. threads > 1 selects the multidev_cc backend (multiple worker-domain CPU devices, for debugging parallel workflows); otherwise the cc backend. Kernel-level CPU parallelism is automatic either way.

Automatically select the best available backend: the backend setting if configured, otherwise the first of metal, cuda, hip, cc whose device discovery succeeds. Only Ir.Backend_intf.Backend_unavailable moves on to the next backend — a driver that is present but fails to initialize, an interrupt, or an assertion failure propagates rather than silently downgrading the run (gh-ocannl-536).

The selection policy of auto, exposed so it can be pinned without a device: true exactly for the failures that mean "this backend is not available on this machine".

Compile assignments into an executable routine. Returns updated context and the compiled routine. The returned context carries the updated compilation frontier for dependency tracking; the input context is unchanged (see execution_deps). name names the routine and its compilation artifacts (generated source files, kernel functions); if omitted, the name is derived from the comp's Ir.Assignments.t.Block_comment labels via Ir.Assignments.get_name_exn, which raises if the comp contains no block comment. lowered_transform rewrites the optimized lowered code before backend compilation — the seam for schedule transforms and for hand-annotating hardware axis types in tests (docs/proposals/axis-types-for-loops.md). It returns the routine's kernel segments, so a whole-routine transform returns a singleton (fun o -> [ f o ]) and a transform that splits the routine into several kernels (fission) returns one element per segment; the segments run back-to-back on the routine's stream with device-side events at the boundaries, like Ir.Schedule.maybe_default_schedules' segments. It must return a non-empty list. On a GPU backend the kernel list -- the transform's, or the default schedules' without one -- then goes through zero elision (gpu_elide_covered_zeros, gh-ocannl-1257): a covering zero whose node an accumulation next covers is dropped, and the accumulation opens from zero itself, with values unchanged.

prelowered (gh-ocannl-562) is a test seam, exempt from that elision: it replaces this compile's lowering of comp with the given optimized code, which then drives codegen AND the analysis layer (I/O classification, liveness planning, the context-buffer delta) alike — so a hand-built Ir.Low_level.optimized can be seeded with set_values, run, and read back with get_values, rather than only analyzed. Pass ~name and Ir.Assignments.empty_comp unless the routine's nodes must settle against a prior context. The record's optimize_ctx becomes the returned context's lineage state, so the caller owns its provenance; production code should not use this parameter.

reuse (gh-ocannl-1526) is a lowering cell (Ir.Lowering_reuse) shared by the compiles of one search: the first compile handed it lowers comp and leaves a snapshot, every later one takes a private copy of that snapshot instead of lowering again, and everything downstream of lowering (the transform, zero elision, analysis, codegen, link) runs as on a fresh lowering. Every compile given one cell must name the same context, comp, bindings and name; another raises Invalid_argument. Exclusive with prelowered.

Refuses a released context with Ir.Utils.User_error, raised rather than returned as an outcome by compile_outcome as well. Likewise refuses, before lowering anything, a comp owning a tensor fragment (Ir.Assignments.owned) that a routine of this context's compilation lineage already owns (gh-ocannl-1724), naming each fragment and that routine: a fragment is compiled into one routine of a lineage -- a second copy of a backprop fragment would double-count a gradient. Re-running the earlier routine is run's job; an alternative compile (another lowered_transform, a placement arm) is compiled as a sibling, from a context that does not descend from the owner. The lineage records a routine's fragments only once its link succeeds.

Also refuses, the same way, a comp whose tensor-generated backprop accumulates into a gradient (Ir.Assignments.accumulates) that the lineage cannot keep correct (gh-ocannl-1744), naming the gradient and both routines: Tensor gives a shared operand's gradient zeroing (and its backprop) to the operand's first-constructed consumer only, while every consumer accumulates into it -- so two Train.grad_updates over one Require_grad input, parameter or trunk give neither loss's gradient. Refused when another routine of the lineage accumulates into the gradient, unless neither of the two zeroes it (Ir.Assignments.zeroes) and every zeroing of it in the lineage -- at least one, accumulating nothing into it -- was compiled before the first accumulation; and when an earlier routine of the lineage already carries the backprop propagating FROM the gradient (its contribution would never propagate: compile contributors first, the propagating routine last). The supported spellings: one routine over the objectives' sum, or Train.zero_params_grads ~params compiled first and then Train.grad_update ~accum_steps per objective. Hand-written %cd writes to a gradient neither contribute nor reset, siblings are alternatives and never collide, and the check is compile-time compatibility only: the order of later runs stays the caller's. Context.join (gh-ocannl-1742) is to lift the refusal by hoisting the zeroing.

Compiled code that a successful compile_outcome handed to its retain callback: the backend's code and the rendering censuses its codegen collected, together with the context it was compiled from (gh-ocannl-1440). It holds no working pools -- those are allocated by a link -- and its code module is reclaimed with it.

Internal containment-aware form of compile. The caller supplies the schedule provenance; public user code should continue to use compile. retain, called once on success before the result is returned, receives the compiled code for a later relink_outcome (gh-ocannl-1440); an exception it raises propagates after the routine just linked has been released.

Links retained code again on the context it was compiled from, as a fresh link of the same code: a new working-pool delta, a new routine id and execution frontier, the same mma, peel, volatility and segments as the compile that produced it, and a link failure classified like the compile's own (phase Backend_link). No codegen and no device compile: this is how a search that validated a cached schedule by compiling it, then released the compile, ships it without compiling it twice. It passes the compile's frontend gate again (gh-ocannl-1301: a tainted session or a discarded node refuses it). At most once per retained value (Invalid_argument on a second call); refuses a released source context like compile. A backend-level code cache composes with it: the relink is the backend's own link.

How many backend compiles have completed on the calling domain (gh-ocannl-1440): each code generation a compile or compile_outcome completes -- lowering through codegen, and the device compile where the backend does it then (cc, CUDA, HIP) -- and each device compile a backend defers to link time (Metal's library compile, done at a code value's first link and reused by a relink_outcome). A compile refused inside its transform, before codegen, is not counted. Monotone; read it as a difference around the work in question.

The analyze-only entry point behind decision_surface (gh-560), generalized over placement decisions (gh-ocannl-514, the placement-space search): the routine's optimized lowering as compile would produce it from this context with materialized decided decide_materialized and inline preferred decide_inline — computed by lowering and optimization alone: no backend codegen, no linking, and no effect on the context (the lineage state is forked like a compile's, the decisions are recorded in the fork, and the compilation frontier and ledger are untouched). Sibling calls for the same routine share the underlying analysis via the analysis cache, so each costs only the specialization replay. Used by Autotune.placement_surface to read the all-materialized specialization of the decision surface, the form Ir.Cost_model.completion_floor bounds partial placement vectors on. It passes the compile's frontend gate first, as does decision_surface: a tainted session or a program over a node the frontend discarded is refused before lowering forces any dims (gh-ocannl-1301, gh-ocannl-1505). facts and limits (default: the context's backend_facts and hardware_limits) are the target the lowering decides for: the facts' blends_value_selects and the limits' online_softmax_auto_block, as a compile on that target would lower. A decision-only reader judging another target than the context's passes its own (gh-ocannl-1780, Autotune.model_target); the optimize context and its placements stay the context's.

gh-560: the analyze-only entry point — the routine's searchable inlining decision dimensions (Ir.Low_level.flip_candidates, one per node, most expensive first) as compile would report them from this context, computed by lowering and optimization alone: no backend codegen, no linking, and no effect on the context (the lineage state is forked like a compile's, and the compilation frontier and ledger are untouched). Sibling compiles of the same routine share the underlying analysis via the analysis cache, so after a compile this costs only the specialization replay. Used by Train.tune_placements' flip refinement to read the decision surface.

Execute a compiled routine. Mutates buffers in-place. Returns updated context with newly initialized nodes tracked. Raises Failure if execution dependencies are not satisfied, and Ir.Utils.User_error if t was released (or a lifecycle ancestor of it was), or the routine's compile context was and t is not a live upload fork of it (a fork reference-counts the routine's buffers), or if a required input was never initialized. A read-before-write input — one the routine reads and then writes, such as a counter or a running statistic — must have been written by something in this lineage (a run, an upload, a device copy): its allocation-time zeros do not count (gh-ocannl-1339). State meant to start from zero declares that in its own computation (an inline declaration, embedded in the routine) or is seeded with set_values. A read-only input must have been written the same way -- the buffer the routine binds, not merely the node in some context -- unless allocation-time zero is declared its correct value (Ir.Tnode.mark_zero_valid; every gradient is marked at creation, being an accumulator), so a forgotten data input is refused rather than read as zeros (gh-ocannl-1462).

Before any of that, the routine's requires (gh-ocannl-1724): each backprop fragment its gradient writes flow into must be owned by a routine of its lineage, or nothing would ever propagate from that gradient (a silent drop: the unread write is dead code). The lineage is t's when t descends from the routine's compile (same root context, the routine among t's compile ancestry), so an owner compiled after the routine counts; otherwise the routine's own compile context's -- the routine and its ancestors. A routine compiled in a sibling never counts, even one that ran: a sibling is an alternative, not a part of this computation. The refusal (Ir.Utils.User_error) names the fragment and any owner compiled outside the lineage; the fix is to compile one routine from the context the other's compile returned.

A node every routine of a compile only reads (and that is not made context-owned by Ir.Tnode.set_context_owned) lives in its device's constant pool, which every independently created context on that device shares (the routine's shared_reads). Each such context keeps its own ledger, so a write one of them makes there would silently change what another's routine reads; run therefore also refuses a read of such a buffer that another root wrote after this lineage last wrote it, or adopted it by a copy (gh-ocannl-1623). Within one root its compile siblings share the buffer as before.

The routine's static-index bindings are read HERE, not when the device gets to the work: the dispatch carries the values they hold at this call, so a caller may rebind them for the next run the moment this returns, without syncing, on every backend. That is what makes the bind/run/ rebind loop — Train.sequential_loop, and every training loop — mean the same thing under an asynchronous scheduler as under a synchronous one. Execution itself may still be asynchronous; sync, or a host read, is what waits for it.

Last, a gradient that accumulates across runs with no reset (gh-ocannl-1781). A routine whose computation accumulates into a gradient's context buffer without zeroing it (Ir.Assignments.accumulates but not Ir.Assignments.zeroes) is refused, with Ir.Utils.User_error naming the gradient and any routine of the lineage that zeroes it, once it has run as many times as the computation's Ir.Assignments.accumulation_runs allows (one by default) since the buffer's last reset: a run of a routine zeroing it, or a write from outside a run (from_host, set_values, copy) -- zero or not, a deliberate value. Each further run would add another step's gradient to the sum of the previous ones. Counted per routine and per concrete buffer, so routines contributing once each per reset stay legal; the opt-in for deliberate accumulation over k micro-steps is Train.grad_update ~accum_steps:k. ~timing:true (the autotuner's repeated timing dispatches) waives the refusal; its runs still count.

save_accumulation_history ctx captures the gradient-accumulation history run keeps for ctx's lineage (gh-ocannl-1781), and returns the function setting it back. For a caller that restores buffers it saved itself -- the autotuner's timing-state restore (gh-ocannl-1621) -- so that the restoring uploads, and the runs they undo, are not taken for a reset.

run's pre-dispatch validation on its own — poisoned lineage, uninitialized inputs, unsatisfied execution dependencies, out-of-range bindings — raising exactly what run would, short of its gradient-reset refusal (gh-ocannl-1781), which depends on the run history. All of it precedes the dispatch, so a failure here proves nothing was executed and no device buffer was written. That is the point (gh-ocannl-550): a caller running a routine inside a launch-tagged failure boundary validates through this rather than letting an unattributable failure at Launch condemn the lineage for a mistake nothing had yet acted on (gh-ocannl-564). Equivalent to check_lineage_runnable followed by check_launch_bindings; a caller that must treat the two halves differently calls them directly.

The lineage-wide half of check_runnable: poisoned lineage, uninitialized inputs, unsatisfied execution dependencies. Every one is a property of the context and the computation rather than of any particular compiled form, so a genuine failure here fails every candidate an autotuner could compile in this lineage, and is the caller's to fix and retry.

A search must therefore let this one out (gh-ocannl-569). Contained as a per-candidate decline it is silent: on a backend whose serial baseline is not dispatched — every GPU backend, gh-ocannl-532 — every candidate declines for this one reason, nothing is timed, and Train.tune_placements ships the untuned default out of an unusable lineage under a report that says the search completed.

The per-candidate half of check_runnable: bind-time validation of the launch parameters the caller just wrote (assigned at all — gh-ocannl-940 — non-negative, within the declared static range, within the index width). Candidates differ in their static ranges, so one can fail this while its siblings time cleanly — which is what makes it the half a search contains as a decline (gh-ocannl-564).

Blocks until the context's device is idle. Host reads (to_host, get_values) synchronize on their own; explicit sync is for timing runs (e.g. the autotuner) and for fencing against out-of-band observation. Refuses a released context with Ir.Utils.User_error.

The backend's own failure classifier, for callers that wrap run / sync in Ir.Schedule_outcome.protect. compile_outcome passes it in itself; launch and sync are raising APIs, so the autotuner obtains it here. Passing a classifier that always answers None (as the timing loop did before gh-ocannl-536) makes every launch failure fatal by phase default, with no way for a backend to declare one its candidate's fault.

Undoes run's execution marking for the given routine id. run marks a routine executed before the later sync can report an asynchronous failure, so a contained launch/sync rejection has to withdraw that claim — otherwise the next routine compiled in this lineage waits on a dependency that never completed. Only sound when the failure is known not to have written device buffers; otherwise use poison_lineage.

Marks this execution lineage unusable because a failure may have left device buffers partially written. Every subsequent run, sync, to_host and from_host on any context sharing the lineage raises, naming the routine and the original failure. There is deliberately no restore: recovering would mean rebuilding inputs and parameters, which the current timing_ctx-shaped API cannot express (gh-ocannl-536).

The failure that poisoned this execution lineage, if any — the exception every entrypoint on it now raises. Lets a caller that would otherwise start fresh work on the lineage see that it cannot run: Train.tune_placements checks it before searching the second placement arm, since the arms share a lineage and a poisoned one refuses every timing run (gh-ocannl-550).

The backend's own dump of the properties of all its devices — names, ordinals, and the queried attributes each backend chooses to surface, including the launch-dimension limits hardware_limits derives from. Not redundant with hardware_limits: what a gate compares against may be a device query on one backend and an architectural constant on another, so the raw props do not tell you what the gate uses and the derived limits do not tell you whether the underlying query answered. Printed together by bin/device_props.ml (gh-ocannl-684).

One Sexp.message-shaped device entry per device under a group atom, the same shape on every backend: see Ir.Backend_intf.parse_static_properties, which states the contract and is the single reader of it (gh-ocannl-710).

Concrete per-device capabilities for persistent timing evidence, separately from conservative construction limits. CUDA/HIP include model, architecture, compute resources and static memory/ clock capabilities. CPU includes its model/host and existing compiler fingerprint; Metal adds host hardware UUID and device registry identity. Observed toolchain metadata (CPU compiler, macOS build, HIP runtime/HIPRTC) is optional and intentionally incomplete: CUDA driver versions, selected OpenMP libraries and HIP headers remain provenance gaps. This API does not promise a globally complete performance-environment fingerprint. Outer None means device discovery failed and disables both cache stores; absent toolchain metadata alone preserves persistence.

The backend's conservative per-workgroup device limits (all-None on backends that do not bind hardware axes). Chiefly for schedule transforms and the autotuner.

Stable facts from the selected backend's C-syntax configuration. Use the compiled routine's censuses for decisions that depend on a particular lowering or rendering.

The selected backend's capability facts (Ir.Backend_intf.backend_facts, gh-ocannl-1723): what the scheduler, the autotuner and the schedule cache ask of a backend instead of testing its name. Static; opens nothing.

The facts of the backend Backends.get_backend parses the name as -- for code that holds a backend name rather than a context, such as a test reading the backend setting. Raises Invalid_argument on a name get_backend does not accept: a name is parsed, never classified by what it contains.

Whether the two contexts are on the same backend (typed identity, not name or facts equality: two backends stating equal facts are still two backends).

Whether all execution dependencies of the routine have been satisfied (i.e., all prerequisite routine IDs have been executed).

Copies the node's device buffer from src into dst (default ~into_merge_buffer:No), or into dst's stream's merge buffer for ~into_merge_buffer:Copy, returning the updated destination context. When both contexts come from the same backend the copy stays on-device via the backend's device_to_device transfer machinery (for Copy, the source node's latest compiled writer must have an effective executed or externally written value, and the returned context carries both the merge-buffer node and the completed transfer's execution-ledger entry against which the next compile of merge-consuming code is verified). A later Copy refuses while a consumer of the current transfer is pending; after those consumers execute, replacing the slab invalidates their old transfer dependency so they cannot silently re-run against new bytes. A cross-backend copy falls back to a host round-trip (Copy raises). Nodes absent from src's device buffers fall back to the host round-trip as well, serving host-init literals and recomputable virtual nodes (to_host).

A copy into a buffer dst already holds, and a merge-buffer fill, allocate nothing in dst, so the result is the same release unit as dst (as for an upload into an existing buffer): releasing either releases both, and repeated copies do not deepen the lifecycle chain (lifecycle_depth, gh-ocannl-1623). When src and dst already bind the same buffer (two independently created contexts sharing a device-wide host literal, or two contexts of one root sharing its constant), nothing moves: dst's ledger adopts the value -- and its current write token -- only if src's ledger vouches for it and no other root has overwritten it since (see run).

Whether the context's execution ledger (shared by its root) records the context's buffer for the node as written -- by a run, an upload or a device copy: what run asks of its inputs that are not Tnode.zero_valid. false for a node with no buffer in the context. Readable on a released context: the release retired the records of the buffers it freed, so a released leaf's node reads false unless a later write reused its location (gh-ocannl-1738). A diagnostic.

How many per-id records the context's execution ledger (shared by its root) holds: routine names, executed ids, merge-buffer reader registrations and per-run write records. Each compile adds its routine's; a merge-buffer fill retires its predecessor's entirely, so repeated fills keep the count constant (gh-ocannl-1654). A diagnostic.

How many lifecycle ancestors the context has: the parent links release's refusal of a released ancestor walks at every entry point. A root context is at depth 0; compile and the decide_* recorders add one level, while copy, from_host and set_values add none. A diagnostic (gh-ocannl-1623).

Whether the node has a device buffer allocated in this context.

Whether a host read of a node can be served from a context, and if not, why (gh-ocannl-777).

Classifies a host read of the node from this context without compiling, from the lineage's placements, the context's buffers, the host initialization registry and the stored computations. For a Virtual node the walk follows each stored computation exactly as a replay would read it — the statements the inliner keeps, every operand — into other virtual nodes, and stops at materialized ones. A path ends at the first offending node in program order; when several reasons apply, Unobservable_local is reported over Unobservable_aliased, over Unobservable_merge, over Launch_bound, over Needs_context. Pure: no placement is decided, nothing is compiled. A Recomputable answer describes structure and availability; the recompute's own compile and run keep their checks (e.g. an input no routine of the lineage wrote is refused by run). Refuses a released context and a discarded node like to_host. Raises Failure (an internal error) if a stored computation reads a node this lineage left undecided.

A short placeholder for printers, such as <unobservable: reads Local #7 h>; None for the three readable answers.

The full reason a read of the node from this context is refused, naming the dependency path, the Local node's placement provenance, and the remedy; None for the readable answers.

Returns the node's value in a fresh host Ndarray, classified first by observability:

• Stored: its device buffer is transferred;
• Host_init: a private copy of its host initialization data;
• Recomputable: it is recomputed (gh-ocannl-777). A throwaway routine copying the node into a fresh materialized node is compiled from this context, so its lowering inlines the node's stored computation; it is run, the copy transferred, and its context released (with its ledger records), so the context's buffers, frontier and placements are as they were. A release that fails before freeing raises from the read, and the next recompute retries it. Every such read compiles: no routine is cached (to_host_all shares one routine across nodes). The value is recompute-at-read (gh-617): the stored computation evaluated over its leaves as they are now, which differs from the value the producing routine computed if a leaf was written since. The array has the node's layout; a padded node's fringe holds its committed padding value. The recompute's compile and run keep their own checks: a leaf no routine of the lineage wrote is refused by run, and such a failure propagates, as does any failure after a Recomputable answer.
• any other answer: Ir.Utils.User_error with explain_observability's reason (a node whose buffer the liveness planner aliased; a Local node, including its placement provenance and whether raising a heuristic cap could keep it virtual instead; a node not present in the context; an \@| slice view, whose parent is read instead).

Refuses a released context with Ir.Utils.User_error, as do from_host, copy (either side) and the accessors built on them; and, before forcing anything of the node, a node the frontend discarded (a Tensor.rewind_to_last_closure stale handle; gh-ocannl-1301, gh-ocannl-1505), as do the same entry points.

to_host of each node, in order, with every Recomputable one recomputed by a single throwaway routine -- one copy per node, each inlining that node's stored computation over the leaves as they are now -- so reading n virtual nodes compiles once, not n times (gh-ocannl-1737; Tensor.print_tree reads a tree this way). The values, the layouts and the context left behind are those of the separate reads. Every node is classified before anything is read or compiled, and the first one (in list order) that to_host would refuse raises that refusal. A failure of the shared recompute propagates, as in to_host. A node listed twice is read once: both positions hold the same array.

Uploads the host buffer into the node's device buffer (allocating it if needed) and returns a context in which the node is marked initialized and this explicit write supersedes any compiled reader/writer frontier entry for the node. Raises if this lineage placed the node Local: no routine reads the uploaded buffer.

Retrieves all (unpadded) values of the node via an on-demand device-to-host transfer. The host side of the read allocates the result array and a constant number of heap words more (Ir.Ndarray.retrieve_flat_values).

Sets all (unpadded) values of the node via an on-demand host-to-device transfer, returning a context in which the node is marked initialized.

Retrieves a single value at the given index via an on-demand device-to-host transfer.

Sets a single value at the given index, preserving the other elements. Returns a context in which the node is marked initialized.

Like get_values but extracts a 1d slice of points for plotting.

Like get_values but extracts a 2d slice of points for plotting.

Check if a node is initialized.

Get the name of the backend.

The backend's device ordinal this context runs on -- what cuda/hip/metal was given. This is NOT Ir.Backend_intf.device.device_id, which counts device instances process-globally across all backends; two contexts on different backends share ordinal 0.

(An upper bound of) the memory used for arrays on the context's device, in bytes. Device-wide: covers all contexts sharing the device. Useful for asserting the footprint effect of the liveness memory planner (config buffer_aliasing, gh-ocannl-489).

A CURRENT gauge, so it answers "what is held now" and not "what did this cost": on metal and cc the bytes come back from a GC finalizer, so a reading taken after some work has finished depends on when a collection happened to run. For the footprint of a window of work, bracket it with Ir.Alloc_census.reset_peak and read peak_pool_bytes — a high-water counter raised at the shared allocator seam and never lowered by a free, hence backend-uniform (gh-ocannl-1006).

The driver's (free, total) bytes of the context's device, read now, or None where the backend has no such reading (cc, multidev_cc, metal). Unlike get_used_memory it is device-global: it also counts other processes, the driver's own allocations (contexts, loaded code, kernel local memory, instantiated graphs), and moves in allocation granules -- which is what makes it the instrument for memory OCANNL's pool tables do not see (gh-ocannl-1424).

gh-ocannl-1493: link_delta ctx lowered segments is the device allocation linking a compile from ctx would make, where the compile's lowered_transform received lowered and returned segments -- lowered is the lowering as a transform receives it (zeros already sunk), segments the kernel list (pass [ lowered ] for the unscheduled form). The delta, not the routine's footprint: nodes ctx already holds and constants the device already caches are left out, exactly as the link's own allocator leaves them out, by the same plan. Link-free and hermetic: no codegen, no allocation, no pool id minted, and no effect on lowered, segments or ctx (placement forcing and store fold-back run on copies). Pass ~prelowered:true for a compile given ?prelowered, whose zeros are compiled as written.

It predicts the bytes the link REQUESTS, pool by pool; the driver's page rounding, and memory the driver allocates for itself (loaded code, kernel local memory), come on top. Contrast Memory_budget.footprint, the whole routine's default-schedule footprint, context-free.

gh-ocannl-1493: link_delta, then a trial allocation of exactly the pools it planned, in the order and through the seam the link allocates them (the Ir.Alloc_census records them and Resource_fault_injection.Delta_pool_allocated fires per pool), all freed again before returning, on every path. No memset, upload, constant-cache insert, codegen or link. A device that would refuse the link's allocation refuses this one alike, raising Ir.Schedule_outcome.Device_memory_exhausted -- the driver's own verdict, which no comparison with get_free_and_total_memory reproduces on a unified-memory part. Raises Invalid_argument on a backend that cannot free a pool.

gh-ocannl-1554: a floor under link_delta for EVERY schedule of lowered (the lowering as a lowered_transform receives it) compiled from ctx: the bytes any candidate, cache replay or untuned compile of it must allocate on the device, where link_delta of the unscheduled form bounds none of them from below under buffer_aliasing (one kernel overlaps nothing a fissioned schedule can) or on-chip fusion (a shared epilogue placement keeps its target off the device). Raw node bytes over the nodes of the delta the lowering's code touches, leaving out what some schedule can keep off the device (lf_off_device_bytes) or replace by a packed tile (lf_replaceable_bytes), and under the liveness planner counting the nodes no kernel sequence can overlap plus the largest overlap-eligible node of each precision class (lf_overlap_bytes is what that leaves out). Link-free and hermetic, like link_delta.

The form of a lowered_for_decisions lowering that a compile hands its lowered_transform: the compile's preparation before scheduling (zeros sunk under buffer_aliasing), which moves statements and so fission boundaries. A decision-only reader that runs a compile's transform on a hermetic lowering applies it first, to see what the compile's transform would (Autotune.model_pick, gh-ocannl-1780).

gh-ocannl-1554: link_floor, then a trial allocation of its pools -- each group's bytes in one request, cut only at the per-pool cap -- through the seam the link allocates through (census, Resource_fault_injection.Delta_pool_allocated per pool), all freed again before returning, on every path. Its refusal, Ir.Schedule_outcome.Device_memory_exhausted, answers for every schedule under a page-granular allocator that admits by the total of page-rounded requests -- the floor's requests round to no more than any candidate's pools -- and not against a contiguous-block or fragmentation limit. Raises Invalid_argument on a backend that cannot free a pool.

The context's backend's process-wide debug information (Sexp.message-shaped, backend-specific: e.g. the CUDA and HIP backends' live_streams and live_graph_execs).

Eagerly frees the device buffers this context owns — the pools holding nodes it allocated that its parent does not have, and that are not per-device constants. Idempotent (a second call is a no-op), and safe to call on a context derived from a still-live parent. Compile siblings own separate working pools; upload siblings reference-count their shared pools, freeing each after its last sibling releases it. A linear chain of uploads supersedes its intermediate values: release its latest value, after releasing any compiled descendants. Releasing it also retires the merge-buffer consumers compiled into the chain before the uploads, so they no longer block a refill of the merge buffer; an upload fork keeps them pending until every leaf holding their buffers is released (gh-ocannl-1203). It also retires the ledger's written records (see buffer_written) for every location it gives up -- each pool it frees, and each upload-arena tail it trims, which a surviving sibling's next upload reuses at the same location -- so a reused buffer never inherits a value nobody wrote there (gh-ocannl-1738); records of buffers a sibling still holds stay.

Ownership discipline: release leaves, not interior nodes. A context compiled from this one inherits its buffer locations while keeping it as their backend parent, so releasing an ancestor frees pools the descendant still binds. This is the pre-existing contract of the underlying Backends.finalize. Upload siblings retain their shared pools independently; descendants made by compile or decision recording require their ancestor to remain live (a transfer creates no lifecycle child: see copy). Autotune.tune releases only leaf siblings of one search context.

A context whose buffers were released is a dead handle: compile, run, sync, the transfers and the decide_* recorders refuse it with Ir.Utils.User_error naming the context (gh-ocannl-1329), by the release flag it shares with exactly the values that share its pools (a linear upload chain). A compile sibling or an upload fork has its own flag and keeps working. The same entry points refuse a context one of whose lifecycle ancestors was released (gh-ocannl-1462), naming how many levels up: detecting the misuse does not retain or resurrect the freed pools, so the descendant cannot be used for buffers again, but release itself still accepts it, so it can free its own. An upload fork's lifecycle ancestors are those of the value it forked, not that value, so a fork outliving the chain it forked from is not refused. A descendant of an earlier value of a released linear upload chain is refused even when an upload fork still holds that value's pools, as that value itself is. Nothing in the context is invalidated for reading metadata (placements, names, the ledger, the backend's static queries), only for touching buffers. The check walks the lifecycle chain, one atomic read per ancestor.

Constants (Ir.Backend_intf.field-device.constant_buffer_cache entries) outlive the context that allocated them and are shared: a root's own with its compile siblings and descendants (gh-ocannl-1654), a host literal with every root on the device. Each constant pool is freed, and its cache entries dropped, by the release of the last release unit binding it, whatever its root (gh-ocannl-1762) -- so a loop that builds a fresh graph per step and releases what it ran holds memory flat, and a dropped literal's node, and through Ir.Host_inits its host array, can be collected. A unit nobody releases keeps its constant pools, as it keeps its working ones. Constants are bump-packed, so one live binder of any entry keeps the whole pool.

It frees no ancestor of its own accord, with one exception: the ancestors a wrapper minted on the caller's behalf and handed to the release unit of the context it returns (adopt_ancestor), such as the initialization context of Train.forward_once, are released after the unit's own buffers, by the release of the last release unit holding them (the unit, or the last of its upload forks). A release that fails releasing one of them is resumed by the next release of a value of the unit.

This exists because releasing is otherwise not something a finalizer can be relied on to do (gh-ocannl-550): the backends' pool tables hold a strong reference to every slab they allocated, so device memory is invisible to the OCaml GC and a process under no host-heap pressure never reclaims it. Callers that know an exact lifetime should say so — Autotune.tune does, per candidate. Everyone else can keep ignoring this: nothing regresses by not calling it, the process just holds more device memory.

Whether the two values share a release unit: the same pools and release flag, as the values of a linear upload chain do, or a context and what run returned from it.

adopt_ancestor ctx ~ancestor makes ctx's release unit responsible for ancestor: a lifecycle ancestor of ctx (a context it was compiled or decided from, directly or not) that a wrapper minted on its caller's behalf and does not return, so that no caller could release it (gh-ocannl-1762). release of any value of the unit -- ctx, what run returns from it, a linear upload continuing it -- then releases ancestor after the unit's own buffers, once no release unit holds it: an upload fork of the unit binds ancestor's buffers too and holds it as well, while a context compiled from the unit does not (release leaves first). Several ancestors are released deepest first. Use it only for a context nobody else holds or derives from: releasing it frees the pools every descendant of it binds. Train.run_once and Train.to_routine adopt the contexts they mint. Raises Invalid_argument if ancestor is not on ctx's lifecycle chain, and refuses a released ctx like compile.

The context's compilation lineage's memory-mode resolution (docs/proposals/context-scoped-memory-modes.md): which nodes this lineage decided to inline (Virtual), keep as routine-scoped scratch (Local), or give a device buffer (On_device). Reads are side-effect free; chiefly for tests and diagnostics.

Whether an earlier routine of the context's compilation lineage stored a computation for the node (gh-ocannl-1274): a later compile that reads or updates it replays that inherited prefix, which the routine's own code does not show. Side-effect free.

How many components the lineage's stored template for the node holds: what a later read of it replays, one component per stored setter (gh-ocannl-1653). 0 when none is stored. A diagnostic; side-effect free.

stored_components summed over every node of the lineage's computations table. A diagnostic (gh-ocannl-1653); side-effect free.

A child context whose compilation lineage additionally decides On_device placement for the given nodes: subsequent compiles from the returned context materialize them. This is the functional, context-scoped counterpart of strengthening tnode-level intent (Train.set_materialized) — the nodes' declared intent is untouched and the argument context (with its other descendants) is unaffected, so a default-placement sibling and a materialize-all sibling can coexist (e.g. the placement-A/B arms of Train.tune_placements). Nodes the lineage or intent already constrains away from plain materialization (Virtual, Local, or constant) are skipped.

A child context whose compilation lineage additionally prefers the given nodes inline (gh-555): subsequent compiles exempt them from the heuristic virtualization caps (virtualize_max_visits, virtualize_max_inline_reduction, virtualize_max_inline_fanin) — the caps are priors of the default placement policy, not legality. Legality still applies: a preferred node the virtualizer rejects (escaping symbols, non-injective producer indices, opaque effects, ...) materializes as before, and the observability pessimizations (read-only, read-before-write) are unaffected. The preference steers placements the lineage has not yet decided: a node already materialized by an earlier compile in this lineage keeps that decision (decisions are final within a lineage — compiled routines depend on them), so apply the preference to a pre-compile sibling of the routines that set the node, as Train.tune_placements does. Together with decide_materialized this spans the per-node inlining decision vector: Inline here, Materialize there, the default heuristics elsewhere. Hermetic like decide_materialized: the argument context and its other descendants are unaffected.

A child context whose compilation lineage additionally prefers the given nodes footprint-scoped (gh-ocannl-616), the middle of the placement lattice between decide_inline and decide_materialized: subsequent compiles exempt them from the heuristic caps like an inline preference, and serve their reads from routine-private scratch shaped like each reader's iteration box, filled by a prologue instantiating the node's stored computation over that box right after the producer's last write — the snapshot its materialized buffer would have held; ahead of the reader for a node an earlier routine left virtual — an n-sized scratch and n instantiations for a diagonal reader of an n×n node, against n×n of either for a full materialization or for recompute at n repeated read sites. The default policy lands a node there by itself when a cap would materialize it and the footprint form is strictly smaller; the preference asks for it regardless. Per read site, not a placement: a read the virtualizer cannot serve that way (a guarded read, a read in a shared loop or inside another candidate's template) is inlined instead, legality rejections still materialize, and the node stays Virtual in the lineage with its stored computation. Exclusive with decide_inline per node — each withdraws the other's preference, so the later request wins. Honored only under virtualize_footprint_materialization; same hermeticity and same pre-compile-sibling rule as decide_inline.

Dispatches a reading to its decision: `Materialize to decide_materialized, `Inline to decide_inline, `Footprint to decide_footprint — how a flip of an Ir.Low_level.flip_candidate is applied.

Applies a whole decision vector in the (materialized, inline, footprint) form Ir.Low_level.partition_readings produces and lowered_for_decisions takes: one decide_reading per reading, in the order `Materialize, `Inline, `Footprint. With skip_empty (the default) an empty list is skipped; without it every reading gets its call. The two differ only in context identity, not in the decisions recorded: a decide_reading with [] records nothing, but still derives a fresh child context (a new context id and a backend child counted by the allocation census).