ctak/src, branch mainAn implementation of Tak and a computer opponent in Chttps://git.l-3.space/ctak/atom/src?h=main2026-09-13T19:15:31Zmove data generation into the epoch loop2026-09-13T19:15:31Ztsliltslil@posteo.de2026-09-12T18:37:56Zurn:sha1:7f1566fb21dfe8c99bc770885cc2cc4e478e16b4
Previously we would spend all of the training epochs on making the network a better predictor of the *initial* TD-leaf values that we computed before the start of the first epoch. There's definitely something stale about this, given that the TD-leaf training targets involved the net evaluation at PVs. I'm not overly confident that this makes sense, but as a kind of compromise between fully on-line TD-leaf and the fully stale version we had before, we now recompute the TD-leaf values in each epoch with the fine-tuned net instead of the epoch 0 net.
Perhaps we were just lucky, but this seems to train a strong network under the default setting.
More experimentation is required.
still observing non-monotonic improvements between training generations2026-09-12T14:54:20Ztsliltslil@posteo.de2026-09-12T14:51:16Zurn:sha1:f3ed463e8a215b0ca3219aebbb14c4a7073a0f9fflood fill for roads, fix dragon, memoise leaf eval, enable diagonal in nn_train.py2026-09-12T14:47:15Ztsliltslil@posteo.de2026-09-11T17:36:10Zurn:sha1:a361728f39812e25599323c52510c0eb3131bd90move from single candidate td-leaf with gating to tournament of a population2026-09-11T16:00:18Ztsliltslil@posteo.de2026-09-11T11:56:12Zurn:sha1:6e90f1145b2c5c0f2dcc82d21ec45432274b8cad
The goal in this change was to overcome the rock-paper-scissors phenomenon that was happening in capability space by having on a single candidate learn to beat the most recent version of itself. In some sense this lead to rotating around in capability space instead of chasing monotonic improvements.
I hope that flipping to a system in which many (N=5) candidates play each other and learn from their own set of diverse games in this pool, combined with --replay-iters, will lead to a given network having to have a much more comprehensive understanding of the space of tactics. I haven't yet had time to train this thorougly, will run those experiments on my desktop computer.
credit Mitchell Riley: we can drop 65 parameters for about the same strength; also train for longer2026-09-10T20:04:40Ztsliltslil@posteo.de2026-09-10T13:12:41Zurn:sha1:220a4b79e4b6d742a9ca7c278a1962b870500fbc
The idea is that the current model is M_theta(board, ply) = "score for player 0", but we can instead create a different model M'_theta(board) = "score for player 0 assuming that it's player 0's move" and for a certain pair of functions sigma, normalise, recover M_theta(board, ply) = sigma(ply) M'_theta(normalise(board, ply)).
This pair of functions is sigma(ply) = +/- 1 (parity), and normalise being a conditional interchange of pieces on the board depending on the parity of the ply. The only things stopping this from being completely straightforward is that nn_train.py evaluates M'_theta itself so we need to pass sigma(ply) to the training loop.
6x6 is still untrained.
use silu instead of relu with the intuition being that we avoid dead neurons2026-09-10T12:54:52Ztsliltslil@posteo.de2026-09-10T11:31:51Zurn:sha1:710d5a5225edb44d171ca9ac6345e9a61aa4f125
In a head-to-head this is +187 elo against the previous weights, although it trained more.
cleanup, match now plays pairs of symmetric games2026-09-07T18:58:03Ztsliltslil@posteo.de2026-09-06T16:24:44Zurn:sha1:7ef68374e8a58684b299b0f46f7ed850d5a87190prevent overflow of the uint8_t ply counter during selfplay or match2026-09-05T16:14:55Ztsliltslil@posteo.de2026-09-05T15:57:17Zurn:sha1:9d0d3a4a34e7c1d2b88929b0e65aa0ffaf65abf8training rework, best weights so far*2026-09-05T15:02:41Ztsliltslil@posteo.de2026-09-05T08:54:11Zurn:sha1:f6ea541ce762b190c7d68f221a24b555ebf0c80b
After quite a bit of experimentation it would appear that a far better way to train on self-play data in this setup is to randomise not the moves but the opening. The new selfplay and match binaries have lost the epsilon parameter and now play a random opening of some length before switching to pure, deterministic engine inference. This also resulted in a cleanup of nn_train.py.
As an added bonus, in this configuration it makes sense to harvest training data from both sides in self-play and consequently the two sides are given distinct game ids (this was almost surely an oversight in the original version of TD(lambda) that i had). I also noticed that rand32 was not using the top 32bits of xorshitf*.
*Under default settings nn_train.py produces a 5x5 weight set that doesn't lose a single game to any other cttei that i cared to look at, when measured using `racetrack` (which has a diverse balanced opening book).
TD-leaf and another negamax bug2026-09-04T20:03:59Ztsliltslil@posteo.de2026-09-03T19:17:22Zurn:sha1:1ca3309dad9d412870b2b793f33af7a7ccabd514
TD-lambda and TD-leaf are separated by the observation that minimax with a parameterised leaf heuristic is differentiable* in the parameters. "All" that had to be done was to extract the "principal variation": which one position survived all of the pruning to become the reported value.
This was a bit fiddly and uncovered a subtle bug (again) in negamax.c: we were not being complete in our overriding of the TT entry and by omitting key it was possible that later searches would find incorrect values.
The more interesting question to me was: how do we introduce enough variety while not corrupting the PV chain? I settled on having selfplay have one of the players be epsilon-random but the other greedy (minimax) and simply report only the greedy portion PV chain. This appears to work well enough in practice and doesn't introduce nonsense training data for the net (we don't force its actions to be nonsensical). I also deciced, perhaps in a departure from tradition, that we had good supervised labels for PVs that happened to be winning states, so the training overrides the targets in that case. Finally rolled back some of the ambition in nn_train.py default arguments.
*almost always