Lachesis found 54 of 68 in-scope CVEs without being told the bug class, vulnerable file, or source location. A further 45 of the 113-case corpus fall outside the reader's modeled sink taxonomy and are excluded from the rate.
The only public blind CVE discovery benchmark of its kind. Hits, misses, and closed coverage gaps remain on the same board.
Rate is over 68 in-scope CVEs; 45 of the 113-case corpus have no modeled detector for the class and are excluded. Closed coverage gaps count as hits; blind performance remains visible below.
The rules were fixed before the score.
Disclosed CVEs across C, Python, JavaScript, and TypeScript whose vulnerable revision is buildable and whose bug class falls within the reader's modeled sink taxonomy.
Each case was analyzed at component scope (the affected functions/module). Absolute rank across a full repository is reported only for repo-scope runs, which are not yet included.
Blind
The tool is never told the bug, its family, or its location.
Whole taxonomy
Enumeration runs over the entire candidate taxonomy, every family.
Region matched
A HIT requires the enumerator to surface, unprompted, a candidate whose file:line falls in the CVE's fixed region.
Reproducible
The reader is open-source; per-case candidate output is publishable and re-runnable.
One score. Every denominator visible.
Breakdowns use the same certified cases as the headline rate.
Blind77.9%
Current79.4%
1 coverage gap has closed without erasing the original miss.
By language
python21/22
c12/17
javascript16/17
typescript5/12
By vulnerability class
injection26/29
memory-safety11/16
other7/11
path-traversal6/8
dos4/4
Patched-revision control armpending
Patched-revision control arm not yet run.
For cases with a patched-revision arm: the sink should disappear or drop sharply once patched.
Every result stays on the board.
Hits link to their evidence path. Misses remain visible with the recorded reason they were not found.
MISS (out-of-catalog category): the real sink is a catastrophically-backtracking module-level regex (_punctuation_re / _simple_email_re) applied by the urlize filter to attacker-controlled text -- a regex-complexity (ReDoS) obligation. No resource.regex candidate fired: the compiled regex constant was not modeled as a resource.regex sink at this scope, so the enumerator had nothing to flag there. Enumeration ran blind over the whole taxonomy (585 candidates) and returned zero class-matched leads.
Coverage gaps stay public when they open and after they close.
59 open1 closed
Open gaps
1–6 of 59
Gap openCVE-2020-28493
out-of-catalog sink category
MISS (out-of-catalog category): the real sink is a catastrophically-backtracking module-level regex (_punctuation_re / _simple_email_re) applied by the urlize filter to attacker-controlled text -- a regex-complexity (ReDoS) obligation. No resource.regex candidate fired: the compiled regex constant was not modeled as a resource.regex sink at this scope, so the enumerator had nothing to flag there. Enumeration ran blind over the whole taxonomy (585 candidates) and returned zero class-matched leads.
Gap openCVE-2023-50783
out-of-catalog sink category
MISS (out-of-catalog category): the defect is authorization bypass (decorator-mediated). The defect is the wrong permission on an endpoint -- an access-control decorator, not a data-flow sink. The 16 emitted sink families model dangerous data-flow SINKS (memory copy/alloc/index, injection exec/query/markup/format/document, filesystem path/temp, navigation fetch/redirect, object-integrity deserialize/prototype, crypto primitive/transport, resource regex) -- they do NOT model missing access-control checks, protocol/state ordering, uninitialized reads, NULL dereference, resource exhaustion, sandbox-boundary integrity, or logic/type confusion. So no candidate can match the real defect. No pipeline was run because the verdict is fixed by the catalog: the class is not one of the modeled families regardless of the graph.
Gap openCVE-2017-12448
sink shape not modeled
MISS (temporal derivation gap -- family not emitted): the defect is use-after-free, which maps to the temporal lifetime family mem.lifetime.* . That family is blocked by two unbuilt pieces (task #45 interprocedural stitcher + per-function convergence gate; task #46 cross-TU free/alloc wrapper resolution), so no candidate can surface today. The wrong release routine frees a bfd still traversed by the nested-archive walk in a different function. The 16 emitted sink families are spatial/injection/traversal; temporal typestate (use-after-free / double-free) is not derivable yet -- a matcher/stitcher gap, not a sink-recognition claim. No pipeline was run because the verdict is fixed by the catalog: the class cannot be emitted regardless of the graph.
Gap openCVE-2025-66489
out-of-catalog sink category
MISS (out-of-catalog category): the defect is authorization / logic. The defect is a broken auth conditional that skips password verification -- an authentication-logic bug, not a data-flow sink. The 16 emitted sink families model dangerous data-flow SINKS (memory copy/alloc/index, injection exec/query/markup/format/document, filesystem path/temp, navigation fetch/redirect, object-integrity deserialize/prototype, crypto primitive/transport, resource regex) -- they do NOT model missing access-control checks, protocol/state ordering, uninitialized reads, NULL dereference, resource exhaustion, sandbox-boundary integrity, or logic/type confusion. So no candidate can match the real defect. No pipeline was run because the verdict is fixed by the catalog: the class is not one of the modeled families regardless of the graph.
Gap openCVE-2026-23478
out-of-catalog sink category
MISS (out-of-catalog category): the defect is authorization / logic. The defect is trusting client-supplied session email in a jwt callback -- an authentication-logic bug, not a data-flow sink. The 16 emitted sink families model dangerous data-flow SINKS (memory copy/alloc/index, injection exec/query/markup/format/document, filesystem path/temp, navigation fetch/redirect, object-integrity deserialize/prototype, crypto primitive/transport, resource regex) -- they do NOT model missing access-control checks, protocol/state ordering, uninitialized reads, NULL dereference, resource exhaustion, sandbox-boundary integrity, or logic/type confusion. So no candidate can match the real defect. No pipeline was run because the verdict is fixed by the catalog: the class is not one of the modeled families regardless of the graph.
Gap openCVE-2018-1000030
sink shape not modeled
MISS (temporal derivation gap -- family not emitted): the defect is TOCTOU race -> use-after-free / heap overflow, which maps to the temporal lifetime family mem.lifetime.* . That family is blocked by two unbuilt pieces (task #45 interprocedural stitcher + per-function convergence gate; task #46 cross-TU free/alloc wrapper resolution), so no candidate can surface today. The buffer free and the concurrent read are in separate threads with no lock; the reuse is temporal across threads, not a single-path sink. The 16 emitted sink families are spatial/injection/traversal; temporal typestate (use-after-free / double-free) is not derivable yet -- a matcher/stitcher gap, not a sink-recognition claim. No pipeline was run because the verdict is fixed by the catalog: the class cannot be emitted regardless of the graph.
HIT (SAST-native sink surfaced by blind whole-taxonomy enumeration once the receiver-tainted regex sink was catalogued + tool-witnessed subject provenance)
Whether blind enumeration surfaced a candidate inside each CVE's fixed region.
Scope varies
Each case was analyzed at component scope (the affected functions/module). Absolute rank across a full repository is reported only for repo-scope runs, which are not yet included.
The denominator
The rate is over the 68 in-scope CVEs. 45 of the 113-case corpus are excluded because the reader has no applicable detector for the class: 26 whose sink class the catalog does not model, and 19 temporal or lifetime bugs (use-after-free, double-free) whose family the reader does not yet emit. In-scope misses, where a detector ran and missed, still count against the rate.