SyScan 2014, re-read in 2026: measurement beats costume

אִם יִרְצֶה הַשֵּׁם

In April 2014, at SyScan Singapore, dyb gave a talk titled Scientific Best Practices for Recurring Problems in Computer Security R&D. It is a very 2014 deck in places: CAN-bus hardware demos, GoodThopter boards, Mark Stoeffinger’s magnetic side-channel work at NTU, MINE/MIC statistics, Georg Wicherski’s ROPe detection via performance counters, Bochspwn kernel instrumentation, and “cyber mission planning.”

Re-read in 2026, the strange thing is not how dated it feels. It is how much of the argument survived.

The core claim

The deck’s real thesis was not “use this CAN tool” or “trust MIC statistics.” It was: security research should be run like a measurement discipline. Find observable channels, represent them honestly, detect structure without fooling yourself, verify across independent signals, and only then convert the result into mitigation or operational effect.

The talk’s three practical questions were blunt:

1. How do you represent higher-dimensional live signals?
2. How do you detect them without drowning in type I/II error?
3. How do you prevent or mitigate the leak once you can see it?

That was a good argument in 2014. By 2026 it looks almost conservative.

Automotive security went from hobbyist to compliance

The opening addendum was deliberately low-budget: a small team, cheap tools, Python glue, a GoodThopter, a couple of cars, and undergraduate students who learned reverse engineering quickly enough to manipulate dashboard CAN data. The point was methodological more than theatrical. Car bus compromise was reproducible, cheap, teachable, and boringly systemic.

The industry response eventually became boring in exactly the way that framing implied. UN Regulation No. 155 now requires a certified Cyber Security Management System for vehicle type approval, and ISO/SAE 21434 supplies the engineering vocabulary: item definition, asset identification, threat analysis and risk assessment, cybersecurity goals, controls, verification, and post-production monitoring. The EU/UNECE application timeline moved from new vehicle types in July 2022 to all new vehicles in July 2024.

The 2014 demo was not “wrong” because cars later added gateways, authenticated diagnostics, secured boot, IDS signatures, key management, OTA patching, and supply-chain assurance. It was right because those mitigations were always going to be socio-technical, not purely cryptographic. Once a message bus becomes safety-relevant and long-lived, the attack surface stops being a single CAN frame and starts becoming a lifecycle.

The best 2026 version of the lesson is: adversarial reproducibility beats ceremonial compliance. A CSMS can still be paper-thin if nobody is allowed to say, plainly, “we injected this ID and the dashboard lied.”

Side channels became the center, not the fringe

The strongest part of the deck is its treatment of side channels as evidence. It drew on cache hierarchy, MMU/TLB behavior, timing, power, EM radiation, acoustic signatures, OS events, and database/protocol timing attacks. In 2014 this still read as a specialist annex to “real” vulnerability research. After Meltdown and Spectre in 2018, that hierarchy collapsed. Microarchitectural state became a primary security object.

This is the biggest validation. The 2014 slide citing Hund-style MMU/cache timing attacks looks ordinary today because speculative execution, transient-execution attacks, cache occupancy, branch predictors, page tables, prefetchers, and thermal/power leakage are now part of ordinary threat modeling. The novelty has moved from “can timing reveal secrets?” to “which hardware state transitions are observable, by whom, under what attacker model, and what is the cost of closing or randomizing the channel?”

The “consilience of induction” framing (Whewell’s aggregate-evidence idea, rendered as a duck test) also aged well. Modern side-channel work is rarely convincing because one sensor squeaks. It is convincing because timing, performance counters, power traces, instruction retirements, cache misses, and architectural outputs converge. A leak is a hypothesis about shared physical state, not a vibe.

The statistics lesson: useful, but not magic

The deck was bullish on MINE/MIC statistics for detecting general relationships in noisy higher-dimensional data. That is the part dyb would annotate most heavily in 2026.

MIC was attractive because linear correlation is blind to ellipses, thresholds, mixtures, and weird functional forms. But the “equitability” story was quickly challenged. Kinney and Atwal’s PNAS comment argued that MIC’s equitability claims did not hold as advertised, and simulations showed MIC could have relatively low power for many relationship types, sometimes losing to simpler estimators such as mutual information, distance correlation, or Hoeffding’s D.

The fair 2026 reading is: MINE/MIC is a useful exploratory screen for suspicious dependence, not a universal dependency oracle. In security telemetry, the real work is not finding a pretty scatterplot; it is separating a real signal from correlated infrastructure noise, staged updates, cron jobs, benchmark jitter, thermal throttling, and attacker-induced baselining. The best modern analogue is less “run MIC and believe the red heatmap” and more “use multichannel change-point detection, invariant testing, causal-ish interruption tests, and adversarial replay.”

The emphasis on representation and detection remains correct. The specific statistical mascot aged the least well.

Bochspwn-style instrumentation won quietly

Bochspwn (Mateusz Jurczyk and Gynvael Coldwind’s whole-system memory-access analysis) looks prescient in hindsight. The original work used CPU emulation and instrumentation to find Windows kernel race conditions and double-fetch-style bugs; Google’s later summary credits Bochspwn with finding over 50 conditions fixed across MS13-016/017/031/036. The “Reloaded” era in 2017–2018 used similar whole-system pattern analysis to identify over 70 Windows kernel bugs and more than ten Linux kernel bugs.

The durable idea is bigger than Bochs: make illegal or improbable memory behavior observable at scale. Today that lineage shows up in kernel fuzzing, KASAN/KMSAN/UBSAN, syzkaller, sanitizers, concolic execution, taint tracking, Arm/Intel tracing, VM introspection, and exploit-primitive-aware crash triage. The lesson for 2026 is: a bug class matters more when you can make its precondition visible before it becomes a write-up.

ROPe and mission planning: right instinct, different vocabulary

The ROPe / cyber-mission-planning material is the most period-specific. ROPe, in the SyScan neighborhood of Georg Wicherski’s Sandy Bridge work, was not “ROP gadgets are cool.” It was detecting kernel-level ROP through branch-return mispredictions: shadow-stack intuition expressed through PMCs, LBR checks, and a spectral signature of weird control flow. Return-oriented programming did not disappear, but the ecosystem hardened around it: CFI, shadow stacks, ARM Pointer Authentication, Intel CET/IBT endbr64-style gadgets, hardened allocators, kernel mitigations, and increasingly exploit-agnostic detections.

The interesting 2016–2026 shift was from “chain gadgets” to “make data-only and post-exploitation behavior expensive”: control-flow integrity plus token theft resistance, sandboxing, eBPF visibility, telemetry, and persistence hunting.

So the deck’s instinct (score exploits by operational consequence, not by novelty) was ahead of the compliance curve. The vocabulary was not. Modern equivalents include attack-path analysis, exposure management, breach-and-attack simulation, purple-team validation, cyber ranges, and adversary-emulation coverage mapped to MITRE ATT&CK. The enduring question remains: What does the adversary get, for how long, at what detection cost?

What a 2026 edition would keep, change, and add

Keep

• side channels as measurable evidence
• multi-signal convergence before declaring truth
• cheap adversarial reproduction as a forcing function
• verification debt as a real cost
• explicit attacker model and operational effect

Change

• replace MIC enthusiasm with a toolbox: change-point detection, independence tests, robust statistics, replay, controlled experiments, counterfactual baselines
• move beyond dashboards into queryable evidence graphs
• treat compliance artifacts as hypotheses to be attacked, not deliverables to be admired

Add

• speculative / transient execution and microarchitectural leakage as a default threat class
• supply-chain and SBOM-driven compromise paths
• post-quantum migration as a signal-to-noise and inventory problem
• AI-agent tooling: prompt injection as a side channel of intent, provenance as evidence, evaluation against adaptive evaluators, and agent-loop verification debt
• cloud / edge / kernel / hardware co-design, where a “bug” may be a misunderstanding among firmware, hypervisor, driver, and silicon errata

Final verdict

The 2014 deck stood the test of time better than most conference talks because it was a methodology wearing a tools costume. The CAN addendum became automotive cybersecurity regulation. Side channels became the dominant hardware-security story of the late 2010s. Bochspwn-style instrumentation became ordinary engineering hygiene. MINE/MIC became a cautionary tale about overclaiming generality. ROPe gave way to exposure management and adversary emulation.

Grade in 2026

The punchline is almost embarrassing: the field still has not fully absorbed the deck’s most useful lesson. We generate more signals than ever (EDR, firmware, SBOMs, cloud audit logs, hardware counters, model traces) and we still too often mistake correlation in a pretty panel for evidence. The duck test would not be impressed. It would ask for the genome, the Pope, the water samples, and the timing trace.

Source

• dyb, Scientific Best Practices for Recurring Problems in Computer Security R&D, SyScan Singapore, 3 April 2014. PDF (Internet Archive)