Module Ocannl.Train

Sets the tensor's value as materialized (device-resident, inspectable on demand via the context), and returns the tensor's forward code with a label-derived comment.

A scalar non-differentiable accumulator for grad_update's ?accum_loss: zero-initialized at allocation and materialized. Read it with Context.get_values (which awaits the device) and reset it with Context.set_values ctx t.value [| 0. |] — e.g. once per epoch.

Returns the tensor's forward, zeroing gradients, and backprop code wrapped with label-derived comments. Sets the tensor's value as materialized. If setup_for_parallel is true (false by default), sets the parameters and their gradients as "non-local" (on-device). When accum_loss is given (see loss_accumulator), the update also accumulates the loss value into it (accum_loss =+ loss): training loops can then read the loss sum once per epoch instead of once per step — on GPU backends a per-step Context.get_values awaits the whole device, serializing the stream, while steps that only accumulate on device queue up and overlap with host-side scheduling.

See: https://github.com/tinygrad/tinygrad/blob/master/tinygrad/nn/optim.py

All and only bindings with associated ranges are iterated, with the binding's initial value lost. Bindings without ranges remain at their initial values.

Materializes every non-literal embedded tensor node of t (so its value is inspectable on demand via the context). Replaces the old every_non_literal_on_host now that there is no hosted memory mode (gh-ocannl-333).

Placement A/B autotuning: Autotune.tune on comp under the graph's current (default) placements — virtual intermediates plus the compiler's promotions — and again with every embedded node of loss materialized, keeping the measured winner (the arms' best_ms are min-of-N timings on the same device, so directly comparable). By construction the result is at least as fast as the better of the default and materialize-all placements, whichever the search would find; this generalizes the old "materialize everything before tuning" recipe instead of replacing one fixed placement policy with another. Respecting the two-level memory-mode split (docs/proposals/context-scoped-memory-modes.md) — tnode-level memory_mode is declared, semantics-bearing intent, while placement decisions are context-level and functional — the B arm does not touch intent: it tunes from Context.decide_materialized siblings of ctx (and of timing_ctx), so the arms are hermetic and tune_placements leaves no trace on the graph or on the caller's contexts beyond the returned winner. See test/operations/materialize_after_compile.ml. report, when given, observes both arms' reports in order. Other arguments are forwarded to Autotune.tune; the same caveats apply (notably timing_ctx and non-idempotent routines — both arms share timing_ctx's device for their searches).

init_params initializes the parameters of t, via running their forward code or copying from the host as appropriate. If reinit_all is true, all parameters are reinitialized, otherwise only the parameters that are not in ctx.ctx_buffers are initialized.

run_once is a wrapper around init_params that additionally runs code of f t and returns the context. If skip_init is true (false by default), no initialization is performmed. If reinit_all is true (false by default), all parameters are reinitialized, otherwise only the parameters that are not in ctx.ctx_buffers are initialized.

If output_cd_file is true, the global setting output_debug_files_in_build_directory must be true, and the update code is output to a file before shape inference potentially crashes at init_params.

forward_once is a wrapper around run_once that runs the forward code of t.

update_once is a wrapper around run_once that runs the gradient update code of t: both forward and backprop.

printf is a wrapper around Tensor.print that assumes ~force:true, and by default sets ~with_code:false, ~with_grad:true, and ~style:`Default. It takes an explicit context and retrieves values on demand (gh-ocannl-333). If the tensor's value is not already materialized in ctx, it is recomputed via the for_print copy trick so real values are still shown.

printf_tree is a wrapper around Tensor.print_tree that assumes ~force:true, and by default sets ~with_value:true, ~with_grad:true, and ~depth:9. It takes an explicit context and retrieves values on demand (recomputing via for_print if not already materialized).