Oskar Wickström

Oskar Wickström

@wickstrom.tech

Software engineer in rural exile. Autonomous and generative testing, PL, web, and more. Blog at wickstrom.tech, work at antithesis.com

633 followers155 following317 posts

More browser performance work in Bombadil resulting in throughput increase of 2-4x! Any interest in more details? Blog post? Screencast?

Why obsess over throughput? Because faster fuzzing means finding more bugs faster! It's a very important lever for us. It's also fun and addictive. Bombadil ran at ~1 state per second only 6 months ago, and it's now around 100x faster.

Here's Bombadil finding a 1/6^10 probability bug (rolling a die 10 times and finding a particular sequence of numbers) inside Antithesis within minutes.

View full thread

It should be noted that it's a small synthetic benchmark, I'll be back with more results from real-world apps. But combined with last week's bug found by Antithesis+Bombadil, I'm very optimistic about how this plays out.

Here's Bombadil finding a 1/6^10 probability bug (rolling a die 10 times and finding a particular sequence of numbers) inside Antithesis within minutes.

View full thread

I had various designs where if you at least had isolation, then over time you could *maybe* get somewhere. Building up some graph or tree model of the space, balance exploration vs exploitation, etc. The catch is that the code also changes over time, which greatly complicates building useful models.

Here, instead, we have full-system branching time, and the fuzzer uses client-side JS coverage as part of the objective function. It's all done with default properties in Bombadil, no extra config needed. It takes a few minutes to find the bug, it's all online learning, no over-time complications.

Here's Bombadil finding a 1/6^10 probability bug (rolling a die 10 times and finding a particular sequence of numbers) inside Antithesis within minutes.

This was always that one very uncomfortable concession when talking about browser fuzzing: the slowness and the practical impossibility of rewinding time, made it very impractical to do any sort of search through a state space.

That's how I run my wife's site: NixOS, SQLite, bespoke Rust webapp for ecommerce. It's fast and cheap, and ironically it probably has better uptime than the enshittified platforms.

Reading "The Elements of Typographic Style", I got way too inspired and went down a CSS rabbit hole trying to reproduce some of it. Everything doesn't transfer to web. (First image is my version, second is from the book.) Now I want to write something real with this stylesheet!

Bombadil playing terminal Tetris at 900+ actions per second. Can't really see what's going on (replays, menus, etc just flashing by). This is running the full spec language verifier on every frame.

Oskar Wickström reposted

Sorry for not posting for so long... I have been extremely busy working on my new project: The Enchanter (also called Modos Flow): a 300-ppi Eink portable monitor with 60Hz refresh rate and touch screen. It's fully open-source: gitlab.com/zephray/ench....

Maybe I'm dating and deprecating myself in a single stroke, but I genuinely struggle seeing a productivity increase from agentic coding. I try, I retry, like every 2-3 weeks. I always end up tired and slow, either from hand-holding in real time or spending hours undoing the slop.

And this is like a broken record, but yes, some things it does help with (the silly scripts, the CI YAML hell, etc). I just can't seem to get it to hit my quality bar without a lot of effort, which obviously isn't the purpose.

This is not a moral stance. It's not me saying these things aren't sometimes useful, or that they aren't very impressive anyway. It's an observation of the current practical applicability in my overall work. YMMV.

This has now increased 7.8x and 4.6x speedups!

Oskar Wickström@wickstrom.tech

Good stuff landing soon! Bombadil fixed timeouts are going away, in favor of a dynamic strategy that monitors DOM and network activity, in order to react as quickly as possible and not waste time just sitting around. Quick measurements show 2-4x speedups in terms of actions per second.

Good stuff landing soon! Bombadil fixed timeouts are going away, in favor of a dynamic strategy that monitors DOM and network activity, in order to react as quickly as possible and not waste time just sitting around. Quick measurements show 2-4x speedups in terms of actions per second.

I declare myself beaten. Fuzzing btop again and was perplexed why the fuzzer sometimes terminated in the middle of a run. Turns out it used btop to kill itself...