Ir.C_syntaxOCANNL's C-family code generator, shared by the cc, CUDA, HIP and Metal backends: renders lowered routines as C, CUDA, HIP or MSL source through a backend's C_syntax_config, and reports per-compile censuses of the renderings it chose.
The comment opening every kernel's main logic: its statements, or an outlined kernel's part calls.
The ordered XOR lane masks of one warp-shuffle reduction over the power-of-two width.
An ocannl_powi_* exponent's operands: |exponent| as (lo, hi) words, a shift, its sign.
Whether Metal's pooled slot table holds 64-bit entries, as it does under large_models.
The MSL element type of the pooled slot table: ulong or uint, per pool_slot_is_64.
Stack budget for a routine's function-scope Local arrays; heap-capable backends spill the rest.
Stack budget for one pool-parallel Grid chunk's privatized tiles; past it they go to the heap.
The resolved cc_grid_private_bytes_cap: a Grid loop needing more per chunk stays serial.
The default of cc_grid_min_updates, the work floor of a pool-parallel Grid launch.
The resolved cc_grid_min_updates: a launch estimated below it renders serially; 0 admits all.
How many Tile_mma multiply-accumulates count as one scalar update (Cost_model's constant).
val grid_update_count : ?cap:Base.int -> Low_level.t -> Base.intEstimated scalar updates of one launch of a Grid body, saturating at cap.
type mma_rendering = | Mma_intrinsics| Mma_intrinsics_ldmatrixThe intrinsic arms fed by warp-cooperative ldmatrix loads over a Swizzle_b128 staged tile (gh-ocannl-481 item 3) instead of per-lane gathers. Recorded distinctly because the gh-476 sweep must be able to pin which of the two load paths a timing measured.
| Mma_register_tiled| Mma_scalar_fallbackHow one Tile_mma statement rendered: intrinsics, a register tile, or the scalar fallback.
val sexp_of_mma_rendering : mma_rendering -> Sexplib0.Sexp.tval equal_mma_rendering : mma_rendering -> mma_rendering -> Base.boolval mma_census : Base.unit -> (Base.string * mma_rendering) Base.list Base.refThis domain's Tile_mma census: (kernel name, rendering) entries, most recent first.
This domain's census of persistent-fragment scopes the fragment hook rendered, one kernel name each.
val is_tensorized_rendering : mma_rendering -> Base.boolWhether a rendering is a real tensor-core or SIMD-register-tile emission, not the scalar fallback.
type tensorization = | Not_requestedCodegen emitted no Tile_mma statement at all: nothing about this routine claims tensor cores, and a timing of it is not a tensorized timing.
| TensorizedAt least one Tile_mma statement rendered to a real tensor-core or SIMD-register-tile emission (Mma_intrinsics, Mma_intrinsics_ldmatrix, Mma_register_tiled). The C backends' register-tiled rendering counts: it is the SIMD-tile emission those backends have, and the point of the label is fallback-vs-not, not which ISA.
| Scalar_fallbackTile_mma statements were emitted and every one of them declined to the lane-0 scalar fallback. A timing labeled tensorized in this state measured scalar code.
Whether a routine's Tile_mma statements tensorized: none emitted, any honoured, or all fell back.
val sexp_of_tensorization : tensorization -> Sexplib0.Sexp.tval equal_tensorization : tensorization -> tensorization -> Base.boolval tensorization_name : tensorization -> Base.stringThe label a census line prints for a tensorization state.
type mma_summary = {renderings : (Base.string * mma_rendering) Base.list;The census entries of one compile, in emission order (kernel name, rendering). Fissioned segments of one routine contribute their kernels to the same summary.
*)tensorization : tensorization;statements : Base.int;Tile_mma statements emitted: List.length renderings.
scalar_fallbacks : Base.int;Of those, how many rendered as the lane-0 scalar fallback.
*)fragment_scopes : Base.int;gh-ocannl-1707: persistent-fragment regions (Low_level.fragment_region) the backend's fragment hook rendered, the accumulator register-resident across the whole reduction. Not counted: a region whose hook declined (its Tile_mmas alias the target per statement, or fall back), and a pad-masked contraction's fragment, which is never marked as one.
}What a compile's mma_census says about the routine it produced.
val sexp_of_mma_summary : mma_summary -> Sexplib0.Sexp.tval summarize_census :
?fragment_scopes:Base.int ->
(Base.string * mma_rendering) Base.List.t ->
mma_summarySummarize census entries and the fragment_scopes count into an mma_summary.
val empty_mma_summary : mma_summaryThe summary of a compile that emitted no Tile_mma statement.
val merge_mma_summaries : mma_summary Base.List.t -> mma_summaryOne summary of several compiles' census entries and fragment scopes together.
val mma_summary_string : mma_summary -> Base.stringThe one-line census a timing report prints: the tensorization label, then per-rendering counts.
val with_census : (Base.unit -> 'a) -> 'a * mma_summaryRun f collecting the Tile_mma census, nesting additively; return its result and summary.
Why a site's width decision declined a base the peel reached: Low_level.accum_skip re-exported.
val sexp_of_peel_skip : peel_skip -> Sexplib0.Sexp.ttype peel_verdict = {levels : Base.int;Loop levels the localized scope spans: the level being rendered, plus those Low_level.peel_accum_nest took below it. One for a reduction localized at its own level; two where the accumulated cell let the peel swallow the enclosing level too.
guards : Low_level.peel_guard_verdict Base.list;The verdict of each If among them, outermost first.
}What a localizing site decided: the loop levels its scope spans and its guards' verdicts.
val sexp_of_peel_verdict : peel_verdict -> Sexplib0.Sexp.tval equal_peel_verdict : peel_verdict -> peel_verdict -> Base.booltype peel_site = | Peel_localized of peel_verdictThe site localized, deciding this.
*)| Peel_ceded of peel_verdictThe width decision reached a base the serial rendering would have localized, deciding this — and the site rendered a form that holds the accumulator at that same residency somewhere other than a minted scope: the warp-shuffle tree or the SIMD accumulator grid (gh-ocannl-754). Not Peel_localized, since no scope was minted; not a skip, since the width question was asked and answered wide. A test can therefore pin that the shuffle or the grid took its width from the one shared decision rather than from a recognizer of its own.
| Peel_refused of Low_level.peel_refusalLow_level.peel_accum_nest refused, with why.
| Peel_not_attempted of peel_skipThe peel reached a base, and localization was not attempted, for this reason. Every rendering of the level then keeps the accumulator at storage width (gh-ocannl-754).
*)One reduction site's peel outcome: localized, ceded to a wide rendering, refused, or not attempted.
val sexp_of_peel_site : peel_site -> Sexplib0.Sexp.tThis domain's switch for collecting the peel_census.
val peel_census : Base.unit -> (Base.string * peel_site) Base.list Base.refThis domain's reduction-peel census: (kernel name, site) entries, most recent first.
val is_localized_peel : peel_site -> Base.boolWhether a site localized its accumulator into a minted scope.
type peel_summary = {sites : (Base.string * peel_site) Base.list;The census entries of one compile, in emission order (kernel name, site). Fissioned segments of one routine contribute their kernels to the same summary.
*)localized : Base.int;ceded : Base.int;Sites whose width decision was wide and whose rendering held the accumulator outside a scope (gh-ocannl-754).
*)declined : Base.int;Sites that did not localize or cede, refused and not-attempted together.
*)}What a compile's peel_census says about the routine it produced.
val sexp_of_peel_summary : peel_summary -> Sexplib0.Sexp.ttype accum_width = | Accum_not_a_nest of Low_level.peel_refusal| Accum_base of {tn : Tnode.t;idcs : Indexing.axis_index Base.array;base : [ `Update of Low_level.scalar_t
| `Scope of Low_level.scope_id * Low_level.t Base.list ];debug : Base.string;rebuild : Low_level.t -> Low_level.t;verdict : peel_verdict;What the peel decided, the level being rendered counted in (levels >= 1).
pinned : peel_skip Base.option;Some why: the serial rendering keeps the accumulator in its storage cell, narrowing every step, so no rendering may hold it wider. None: the serial rendering localizes it at the residency, and so may any other.
}How one reduction level's accumulator is held, decided once for every rendering of that level.
type statement_accum = {sa_tn : Tnode.t;sa_idcs : Indexing.axis_index Base.array;sa_op : Ops.binop;sa_contrib : Low_level.scalar_t;sa_verdict : peel_verdict;sa_pinned : peel_skip Base.option;}An accum_width base that is a single accumulation statement at the level itself.
val peel_summary_string : peel_summary -> Base.stringThe one-line peel census: how many sites localized, ceded or declined, and the distinct verdicts.
val with_peel_census : (Base.unit -> 'a) -> 'a * peel_summaryRun f collecting the peel_census, nesting additively; return its result and summary.
type volatility_site = | Volatile_accumulation_reads of Base.stringA reduction-shaped scope local for which the backend selected expression-level volatile pointer casts on any device reads in its accumulating loop, by the local's emitted identifier: the localized accumulation form (gh-ocannl-731, gh-ocannl-820).
| Plain_accumulator of Base.stringA reduction-shaped scope local whose reads stay plain — either this backend does not request the workaround, or no materialized device read was emitted in its accumulating loop. The accumulator itself stays register-resident in both cases.
*)| Volatile_rmw_reads of Base.stringA device-memory read-modify-write at an address invariant across an enclosing serial loop, whose device reads use expression-level volatile pointer casts.
One serial accumulation's volatility-census entry, by the emitted identifier.
val sexp_of_volatility_site : volatility_site -> Sexplib0.Sexp.tval equal_volatility_site : volatility_site -> volatility_site -> Base.booltype volatility_summary = {entries : (Base.string * volatility_site) Base.list;The census entries of one compile, in emission order (kernel name, site). Fissioned segments of one routine contribute their kernels to the same summary.
*)requested : Base.bool;Whether the backend this compile ran on asks for the workaround (C_syntax_config.volatile_serial_accumulation). false makes every localized site a Plain_accumulator and suppresses the device-memory RMW arm entirely.
volatile_accumulations : Base.int;plain_accumulations : Base.int;volatile_rmw_reads : Base.int;}What a compile's volatility census says about the routine it produced.
val sexp_of_volatility_summary : volatility_summary -> Sexplib0.Sexp.tThis domain's switch for collecting the volatility_census.
val volatility_census :
Base.unit ->
(Base.string * volatility_site) Base.list Base.refThis domain's volatility census: (kernel name, site) entries, most recent first.
Whether the backend compiling on this domain requests the volatile-read workaround.
val volatility_summary_string : volatility_summary -> Base.stringThe one-line volatility census: accumulations, and how many use the volatile-read workaround.
val with_volatility_census : (Base.unit -> 'a) -> 'a * volatility_summaryRun f collecting the volatility census, nesting additively; return its result and summary.
type grid_serial_reason = | Grid_opaque_bodyAn opaque statement or a barrier sits under the loop body.
*)| Grid_scope_local_escapesA scope local is set under the loop but declared outside it, so its function-scope storage would race across chunks.
*)| Grid_local_accessed_outside of Base.stringThis local is written grid-variantly (it fails the shared rule) and is also accessed outside the loop body (it fails the privatization rule).
*)| Grid_local_not_covering of Base.stringThis local fails the shared rule, and its first access per iteration is not a standalone covering write (it fails the privatization rule).
*)| Grid_private_over_cap of {}Every local the loop needs privatized could be, but their combined per-chunk footprint private_bytes exceeds cap, the resolved cc_grid_private_bytes_cap (per_chunk_private_bytes_cap).
| Grid_below_min_updates of {}The launch's estimated scalar updates (grid_update_count, saturating at floor) fall below floor, the resolved cc_grid_min_updates.
| Grid_offset_range of {}The loop does not start at 0, which the chunked rendering assumes.
*)Why an outermost Grid loop rendered serially on a pool-backed backend.
val sexp_of_grid_serial_reason : grid_serial_reason -> Sexplib0.Sexp.tval equal_grid_serial_reason :
grid_serial_reason ->
grid_serial_reason ->
Base.booltype grid_site = | Grid_pool of {}Rendered on the pool, each chunk privatizing private_bytes of tiles (0 when none).
| Grid_serial of grid_serial_reasonOne outermost Grid loop's decision: rendered on the pool, or serial and why.
val sexp_of_grid_site : grid_site -> Sexplib0.Sexp.tOne outermost Grid loop: the kernel it is in, its loop index's identifier, the decision.
val sexp_of_grid_entry : grid_entry -> Sexplib0.Sexp.ttype grid_summary = {loops : grid_entry Base.list;The census entries of one compile, in emission order. Fissioned segments of one routine contribute their kernels to the same summary.
*)pool : Base.bool;Whether any kernel of this compile was rendered with pool-parallel Grid loops available at all: a pool-backed backend (cc_parallel_grid not none) with more than one chunk and no debug_log_from_routines. false means no entry was recorded, and on a GPU backend that a Grid loop is a hardware grid, not a serial loop.
pool_rendered : Base.int;serial : Base.int;}What a compile's Grid census says about the routine it produced.
val sexp_of_grid_summary : grid_summary -> Sexplib0.Sexp.tval grid_summary_string : grid_summary -> Base.stringThe one-line Grid census: Grid loops rendered on the pool, and why each other one is serial.
val with_grid_census : (Base.unit -> 'a) -> 'a * grid_summaryRun f collecting the Grid census, nesting additively; return its result and summary.
The address space of a tile-MMA operand as the emission hooks see it.
The physical layout of a tile-MMA operand's storage, which the emission hooks trust as given.
type mma_operand = PPrint.document * Base.int * mma_space * mma_layoutA tile-MMA operand: tile-base pointer, leading-dimension stride, address space, layout.
type mma_source = Base.int * mma_space * mma_layoutA tile-MMA input operand described but not addressed: stride, address space, layout.
The emitting half of an accepted C_syntax_config.mma_syntax call, awaiting the a/b addresses.
type async_copy_syntax = {ac_copy : dst:PPrint.document ->
src:PPrint.document ->
bytes:Base.int ->
PPrint.document;One element-sized asynchronous global→workgroup-shared copy statement; dst and src are element addresses (&ident[offset]) and bytes the element size — 4 or 8 today. The hardware also copies 16, but a 16-byte copy requires a 16-byte-aligned destination, and plain workgroup-shared declarations align to the element type only (4 for uint4x32_t) — so 16 stays out of the per-element eligibility until a rendering guarantees the alignment (Codex P2 on PR #317). The copy is byte-for-byte: eligibility (same storage precision on both sides, no value transformation) is the caller's check.
ac_wait_all : Base.string;Statement completing every asynchronous copy issued so far by the calling thread, committed or not (CUDA cp.async.wait_all). Cross-thread visibility still needs the workgroup barrier the caller emits right after.
}Asynchronous global-to-workgroup-shared copies for staging software-pipelined tiles.
The local a Blend_over_locals blend binds its first arm to; tests find blends by it.
module type C_syntax_config = sig ... endWhat a backend supplies to the C-family renderer: its dialect's spellings, capabilities and hooks.
val codegen_capabilities :
(module C_syntax_config) ->
Backend_intf.codegen_capabilitiesThe codegen capabilities a backend's config publishes, derived from its own hooks.
val default_binop_syntax :
Ops.prec ->
Ops.binop ->
PPrint.document ->
PPrint.document ->
PPrint.documentThe C-family layout of a binop's Ops.binop_c_syntax spelling, shared by every backend.
val rng_binop_syntax :
backend:Base.string ->
call:(Base.String.t -> 'a) ->
Ops.prec ->
Ops.binop ->
'aRender an RNG binop (the Threefry variants, the uniform lane conversion) as its builtin's call.
val all_precs : Ops.prec Base.List.tEvery storage-bearing precision: an alias of Ops.storage_precs.
val operand_conditionality_violations :
ternop_syntax:
(Ops.prec ->
Ops.ternop ->
PPrint.document ->
PPrint.document ->
PPrint.document ->
PPrint.ToBuffer.document) ->
binop_syntax:
(Ops.prec ->
Ops.binop ->
PPrint.document ->
PPrint.document ->
PPrint.ToBuffer.document) ->
select_blend_syntax:(Ops.prec -> select_blend Base.option) ->
Base.string Base.List.tWhere a backend's operator renderings break the operand-evaluation contract; [] when nowhere.
The names a builtins table defines (its entries' keys), for a backend's ident_blacklist.
val builtin_definitions :
(Base.String.t * 'a * Base.String.t Base.List.t) Base.List.t ->
keys:Base.String.t Base.List.t ->
'a Base.List.tThe definitions keys need from a builtins table, dependencies included, in table order.
val vec_widen_macro :
store_prec:Ops.prec ->
prec:Ops.prec ->
lanes:Base.int ->
Base.String.t Base.optionThe cc builtin widening a 16-bit-storage vector to lanes lanes of prec; None otherwise.
val vec_narrow_macro :
store_prec:Ops.prec ->
prec:Ops.prec ->
lanes:Base.int ->
Base.String.t Base.optionThe cc builtin narrowing lanes lanes of prec to 16-bit storage; None otherwise.
The (bits, conv) locals a register tile's partial bf16/fp16 column is staged through.
The identifier prefix ^ base ^ suffix a derivation template spells from base.
Derivation template of a local array's alias pointer.
Derivation template of the allocation a heap-declared local array points into.
val grid_chunk_derivations :
(Base.string * Base.string)
* (Base.string * Base.string)
* (Base.string * Base.string)Derivation templates of a pool-chunked Grid loop's chunk index, lower and upper bounds.
Derivation templates of a lane all-reduce's value and partials.
Derivation templates of a warp-shuffle reduction's value and partials.
Derivation template of an MMA fragment register, applied to its node's uid.
Every derivation template the emitter mints names with, for the kernel code cache's renaming.
val vec_widen_rows_macros :
store_prec:Ops.prec ->
prec:Ops.prec ->
lanes:Base.int ->
(Base.string * Base.string) Base.optionThe (pack, row) cc builtins a register tile's A column widens through; None: per-row widening.
The words C++ reserves beyond C's, for the CUDA, HIP and MSL backends' blacklists.
Whether name lies in the implementation-reserved namespace: __x or _X.
Predefined compiler macros outside the reserved namespace (linux, unix, ...).
type routine_name_probe = {prelude : Base.string;The source every kernel follows: the unconditional includes, with every conditional include ahead of them, so a name any kernel's headers declare is measured.
*)kernel_definition : Base.string -> Base.string;A minimal kernel DEFINITION of the given name, one line, as the backend declares one.
*)compile : Base.string -> (Base.unit, Base.string) Base.Result.t;The backend's runtime compiler on a whole source; Error carries its report.
kernel_ident : Base.string -> Base.string;routine_floor_idents : Base.string Base.list;What routine names avoid besides vendor_routine_idents: the node-name blacklist and the C library's names. The reserved namespace (implementation_reserved) is a rule, not a list.
vendor_routine_idents : Base.string Base.list;The backend's harvested table.
*)}What tools/vendor_idents.exe needs from a backend to harvest its toolchain's file-scope names.
val vector_type_names :
prefixes:Base.String.t Base.List.t ->
scalars:Base.String.t Base.List.t ->
widths:Base.Int.t Base.List.t ->
Base.String.t Base.List.tEvery <prefix><scalar><width> spelling: GPU dialects' vector-type names, for name blacklists.
val diagnostic_scopes :
Base.String.t ->
([> `Error | `Other | `Unscoped ] * Base.String.t) Base.List.tEach line of a compiler report paired with its diagnostic's severity, or `Unscoped.
val routine_name_in_diagnostics :
routine_names:Base.String.t Base.List.t ->
Base.String.t ->
Base.String.t Base.optionThe first of routine_names whose declaration a compiler's error report blames, if any.
The opening of the sentence a compile-failure report blames OCANNL's codegen with.
The one-line codegen-bug sentence, asking for artifact.
Whether a failed compile's report says a header could not be opened: a toolchain fault.
val attribute_routine_name_failure :
?report:Base.string ->
routine_names:(Base.String.t * Base.String.t) Base.List.t ->
Base.exn ->
Stdlib.Printexc.raw_backtrace ->
'aRe-raise a failed compile, as a Utils.User_error naming the routine when its name caused it.
full_printf_support for a %g-capable dialect: true unless prefer_backend_uniformity is set.
Whether source's code tokens (not comments or literals) contain marker's tokens in sequence.
val prepend_conditional_includes :
conditional_includes:(Base.string * Base.string) Base.List.t ->
Base.string ->
Base.String.tPrepend each header whose marker source mentions and that source does not already include.
val with_compiler_options :
compiler:Base.String.t ->
options:Base.string Base.list ->
enrich:(Base.exn -> suffix:Base.String.t -> Base.exn Base.Option.t) ->
(Base.unit -> 'a) ->
'aRun compile, attaching the compiler's options to an exception enrich recognizes.
module Over_locals : sig ... endThe Blend_over_locals renderings, shared by the emitter and operand_conditionality_violations.
module Param_guards : sig ... endBuiltin parameter guards: a test seam checking builtin calls pass arguments without narrowing.
module Source_tokens : sig ... endC source tokens for builtin and header selection: comments and literals are not uses.
module Pure_C_config (_ : sig ... end) : sig ... endThe plain-C defaults of C_syntax_config, the cc backend's, which other backends override.
module C_syntax (_ : C_syntax_config) : sig ... endThe C-family renderer for one backend's config: routine naming, prelude assembly, kernel emission.
module Compile_driver
(_ : functor (_ : sig ... end) -> C_syntax_config) :
sig ... endRenders and compiles routines for a backend, attributing name-caused compile failures to the name.