Swinging to comma.ai's podium

On Sep 10 (little over a month ago), comma.ai (the Linux of vehicle self-driving) published a challenge that consisted of, given their set of components, scoring algorithm and vibe ruleset (will make sense later), contestants needed to come up with the lowest-scoring board. Here is the original post, repo and algorithm:

commaai / PCBGolfAre you better than an autorouter?TeX 60 22

Score = w × h × lPCBA bounding box (mm³) + 50 N_vias + 5,000 N_{copper layers}N = count

The post appeared on my X timeline, and after a few days of having it in my mind, it just begged the question: "What if?" I started working on it on the side, the first submission was on Sep 28. And, while I doubted the readiness of our systems at the time, we got third place with a completely autonomous solution.

Below is how I did this (while being as open as possible), how the contest evolved over time, and my personal conclusions.

Our setup

siliXon consists of a combination of algorithm-based services like our autorouter, autoplacer, validation and manufacturing checks, and an AI model with our custom harness to orchestrate it all. As a company, we've been aiming for a fast, iterative design cycle for our users, which is quite the opposite of the long, end-to-end autonomous design the contest required, so instead of our platform, I had to use our internal systems directly. I also added something else, a frontier model to Rocky, which was in charge of both calling our services and also instructing a Rocky subagent (through our internal Rocky API) for the rest of the work, like fixing connections, moving clusters of components, etc., where our placing and routing couldn't one-shot the problem.

At the time, we only had support for 2-layer boards and top component placement, but our 4-layer and bottom placement support was being finished. It was also a really good opportunity to improve our whole suite of systems on demand in order to build a dense production-focused PCB.

Contest timeline

The comma.ai board basically consists of an MCU, four OBD-C ports with eFuses, a host USB-C port, a 12 V barrel jack, CAN transceivers and a microSD slot. You can play around here to get a sense of the scoring; remember, the lower the score the better.

My first prompt to Rocky was:

“There is this comma challenge: https://github.com/commaai/PCBGolf. That is quite perfect to test our siliXon product in. We want to give it a spin and see if we can take a top place using our products, like our Rocky harness or something!”

We made a bunch of improvements to our systems while working on the challenge in order to achieve our scores as mentioned before:

• Placement: Components on the bottom side, behind a flag. Fixes for connector placement, such as an edge connector facing the right edge but not seated at it. We later updated our whole catalog to natively obtain and maintain each connector's facing direction.
• Fab limits: Our systems use stricter limits than "manufacturable ones", so we set them to JLC’s actual limits for this.
• Router: While improving it, we also added native Metal and CUDA support to speed up iterations.

By Sep 25 we had a board scoring in 4th place on the leaderboard. I was chasing score reductions through vias then:

“It seems like the best win will be through less vias? Let’s work on the router!”

Sep 28 was my first entry: 70,230.

We found this submission wouldn't sustain a real-world power load: the power traces were too thin. Despite this, our submission passed and was added to the leaderboard, which meant something clear: the contest, or at least the testing, wasn't that rigorous. The incentive from comma.ai was hiring interesting individuals, as they often do, a hiring practice that should be mass adopted, honestly.

This made me go over the issues on the contest's GitHub. In issue #3, a comma employee explains that review is “static”: manual, plus some automated checks. Nobody builds the boards, and anything that would need real-world testing gets “the benefit of the doubt”. The examples he gives of what fails are obvious bugs: an overvolted part, a missing CAN channel, overlapping footprints, “failing JLC’s design rules”, and “no 5 mil internal traces for 1A continuous current”.

At this point most submissions were either 2- or 4-layer solutions scoring around 40-70k, so our systems were hill-climbing better placement and routing with 2- and 4-layer variants. But by Sep 30 the contest got interesting again: there was a 26k submission. This was only possible (somehow) with a 1-layer board. And a new issue appeared, where BradyMeighan (the top spot and 26k submission owner) asked whether a board with an empty B.Cu counts as one layer, which got the submission re-scored, decreasing its score even more by halving the layer penalty.

I switched our systems towards a one-layer pipeline (I was only prompting the Rocky orchestrator agent about news and events, like new issues, stuff outside its building efforts). Rocky started to implement things like:

• 0 Ω jumpers and wire links laid on the bare side for every crossing.
• Pins assigned to minimise ratsnest crossings.

The first valid one-layer package was 88 mm and scored 65k for some reason, which would've remained 4th.

On Oct 2 Rocky suggested a big idea: stacking. If the penalty counts copper layers per PCB, three stacked one-layer boards still cost 5,000 in total. Later, comma confirmed exactly that in a new issue (#11): “maximum layers of copper in a single PCB”.

Within the same day we had:

• A level assignment by part height,
• Vertical wire risers between levels,
• Notched levels around the USB-C receptacles,
• An assembly STEP folded to real heights.

Rocky also moved the MCU from LQFP-144 to LQFP-100 with a function-preserving pin remap, which issues #1 and #3 allowed, and switched parts to 0201.

The contest never had a way to get feedback on a submission, or even to know whether it was accepted, so we were forced to ask on the issues manually. They became our roadmap, each one worth a few points or a few thousand:

• #5: Mounting holes aren’t strictly required.
• #4: Horizontal SMT USB-C is fine.
• #6: The LEDs must stay.
• #9 (szykula’s “Design limits”):
  • The 12.85 mm USB-C spacing is mandatory;
  • Assembly “doesn’t have to be to JLC spec” (hilariously bypassing their own rules, again, vibes);
  • A through-hole pad that isn’t a component leg counts as a via;
  • Replacing the barrel jack with an edge-plated finger “has to remain somewhat reasonable I guess”;
  • “If we update this challenge in the future I’d define it a bit more strict”.

We dropped from our saved 23,088 package to 19k and 18k, and submitted. At this point we got first place for the first time, after the 19k submission was validated and the 18k one rejected due to mechanical issues with the connectors (all of this forwarded to Rocky).

By now, after 2 successful submissions, I thought I had an idea of what the validation was, but no, this later changed as well.

2 days later (on Friday; the contest was ending on Tuesday), we pushed even more with a 16k submission, making the barrel jack disappear. The base board itself became a slot with edge-plated walls for the sleeve, and a plated tongue entering the plug's center hole. Once the score updated, the difference between us and second place was almost the same as between 2nd and 5th place.

So I asked Rocky: "There are 3 days left, you think we get first place?" Well, 2 days later I told Rocky "Code red! We need to aim for 13k right now!": there was a new 14,076 submission. That same night we got:

• 14,486: 38 AWG wire links, levels squeezed together.
• 14,251: Better packaging.

At this point I spun up multiple agents in search of ideas like flex, folding, thinner boards and component swaps. Most ideas died on the numbers: the height budget kept pointing at the same three terms. I also had a friend participating in the contest; he mentioned his submissions (which passed and were added to the leaderboard) had DFM thresholds in the red, things Rocky and I were avoiding, so we started relaxing rules too:

• 13,776 (JLC rules): Level 1’s wire links moved onto its top side, level 3 resting on the USB-C shells, plus top-side stitch wires soldered straight onto copper.
• 12,494 at 36.5 mm: Relaxed rules (0.075 mm tracks, 0.2 mm holes).
• 11,961: 0.2 mm FR-4.
• 11,788: Level 3 as 0.07 mm flex.

Most of my submissions got rejected with the reason “your submissions have a 2.95mil trace used for the main power path”, even though they had clearly accepted previous submissions that broke it (vibes, hilarious). The contest only had a few hours left, so I had to adapt either way, and stayed up through the night keeping an eye on Rocky. It re-placed at 36.5 mm with every high-current part on the base level. Widths were sized to the eFuses’ 2.1 A per port: 1 mm and 0.6 mm 12 V copper with pours, 0.35 mm per port. By 05:20, 3 hours before the contest deadline, we had a fully JLC-legal 13,184 submission.

Wrap up

In the last hours, there were 2 new submissions with even lower scores than ours, taking first and second place respectively. And the contest ended!

Here is our submission, and first place:

Third place: siliXon, 13,184 points, its three levels below First place: AnasMalas, 10,345 points

First place (and we assume most other submissions) didn't (nor could) use AI for this job:

Not even frontier models, and in the comparison images above, this is quite easy to spot. On the left, an optimization hill-climb with some cool ideas but mostly cheating the weak scoring system, and on the right, human made, with its "je ne sais quoi" that artificial (or super?) intelligence still does not possess in most fields, even less in electronics. Despite this, siliXon's systems reached third place completely autonomously. I have no electronics knowledge or experience; I'm just a generalist who skipped college and uses specialization's power on demand.

I think this result is pretty representative of the current state and future of digital intelligence. siliXon is, unofficially, the best end-to-end electronics design solution, and at the forefront of this yet-to-be-AI-dominated field.

Thinking about our users, our aim has always been to offer a fast, AI-assisted and iterative experience. We are aware that these autonomous, end-to-end, long but accurate solutions are impressive and enticing, so we are exploring providing frontier models, subagents, and new tiers to reflect and offer this. The future is bright.