I have participated in artifact evaluation committees since 2021. I have done 5 years of CHES, one year of EUROCRYPT, and 2 NDSS years. Over these years, I have seen good artifacts, even great ones, and a fair share of “just OK” ones. There have been no truly terrible ones, though some required a bit of polish.

The approach to artifact evaluation at the different venues also differs, and reviewing an artifact for NDSS this year reminded me of the differences. In the following text, I have some opinions on those differences and on the aims of artifact evaluation at large. I also looked at a how other conferences do artifact evaluation, even if I was not reviewing for them, such as USENIX Sec, CCS, CRYPTO, or ACSAC. I summarize how their approach changed over the years, what badges they use, what they focus on, and whether they have artifact appendices with a strict format. I also look at link rot and availability of artifacts over the years.

I want to highlight the work done by the community at secartifacts.github.io. Having one place that survives in the event of the conference pages disappearing is quite nice.

Focus on reproducibility#

NDSS, as well as a few other conferences (e.g., USENIX Sec), puts a heavy focus on reproducibility. They require that artifacts contain a detailed artifact appendix (USENIX Sec 2026 example). It describes the artifact, its requirements, and setup but most importantly, it contains a detailed description of the major claims of the paper and how they are supported by the artifact and how reviewers can run experiments to verify these claims. The appendix is thus usually very focused on the details of the paper and on how to reproduce them exactly. It certainly helps in reproducing the experiments in the paper.

Last week, I had the pleasure of presenting at Real World Crypto 2026 in Taipei, Taiwan. It was a great RWC in a great place. In this postm I summarize my talk, experience, and takeaways. Plus, at the end, you will find photos of Taipei from my hike to 9-5 peak.

The talk#

I presented our work on security by obscurity in secure hardware (see the slides). Compared to cryptographic theory, which features open design, review, competitions, and sometimes even security proofs, the secure hardware space is very secretive. There is almost no documentation. Much interesting information is gated behind non-disclosure agreements, which dissuade the sort of open work we do as academics and security researchers. Finally, even getting access to newer samples is hard, as they are a controlled object.

Recently, I presented our work on Let’s DOIT: Using Intel’s Extended HW/SW Contract for Secure Compilation of Crypto Code at CHES 2025 in the hot and humid Kuala Lumpur, Malaysia. The slides are available.

My colleague Vojta also presented exciting joint work, episode 2 of our reverse-engineering trilogy: ECTester: Reverse-engineering side-channel countermeasures of ECC implementations. The slides are available.

Few things for future reference on visiting conferences in Malaysia:

I recently attended CHES 2024 and presented a paper on pyecsca: Reverse engineering black-box elliptic curve cryptography via side-channel analysis. Check it out on our site. The slides are available.

While out in Halifax, we decided to enjoy Nova Scotia and went on a road trip. Some tips:

• Peggy’s cove: Kind of cool to cycle through, but not that amazing on its own. Expect lots of tourists.
• Hopewell rocks: Check the tides and the walk around is definitely worth it.
• Cape Breton Highlands: Nice hikes in very remote areas.

Recently we (me & Lukasz Chmielewski) gave a tutorial on pyecsca: Reverse-Engineering ECC implementations Using Side-Channel Analysis at the Summer School on real-world crypto and privacy 2024. You can find the tutorial on GitHub.

Today I came up with a clever solution for a dumb problem. When using multiprocessing or concurrent.futures in a Jupyter notebook one is generally limited to the fork start method. This is because the spawn and the forkserver methods require that the target function is defined in an importable module. If you are working with Jupyter notebooks the target function very likely resides in the notebook itself, which is not importable. So, how do we make it importable?

I recently attended CHES 2023 and presented a work-in-progress poster on pyecsca: Reverse-engineering elliptic curve cryptography implementations via side-channel analysis. Check it out below and also on GitHub.

I recently presented the paper “They’re not that hard to mitigate”: What Cryptographic Library Developers Think About Timing Attacks on Real World Crypto 2022 and IEEE Security & Privacy 2022. If you want to get more info on the paper including a pre-print and additional material, check out its page. The RWC slides are available as well as the IEEE S&P slides.