Module C_syntax.Source_tokens

C source tokens for builtin and header selection: comments and literals are not uses.

type t
val scan : Base.string -> t
val scan_static : Base.string -> t

scan, remembered per domain by content (a bounded number of texts): for text a caller passes unchanged again and again, such as a backend's includes and builtin definitions.

val mentions : t -> Base.string -> Base.bool

Whether the code tokens contain marker's tokens in sequence.

val mentions_indexed : t -> Base.string -> Base.bool

mentions_indexed t answers mentions t for any marker, after one pass indexing t: for many markers over one scan.

val joined : t Base.list -> t Base.option

The scan of the parts' texts joined by newlines, from their scans: None when a part other than the last ends inside a comment or a literal, where the joined text has to be scanned.