credit Mitchell Riley: we can drop 65 parameters for about the same strength; also train for longer

The idea is that the current model is M_theta(board, ply) = "score for player 0", but we can instead create a different model M'_theta(board) = "score for player 0 assuming that it's player 0's move" and for a certain pair of functions sigma, normalise, recover M_theta(board, ply) = sigma(ply) M'_theta(normalise(board, ply)).

This pair of functions is sigma(ply) = +/- 1 (parity), and normalise being a conditional interchange of pieces on the board depending on the parity of the ply. The only things stopping this from being completely straightforward is that nn_train.py evaluates M'_theta itself so we need to pass sigma(ply) to the training loop.

6x6 is still untrained.

1 files changed, 5 insertions, 5 deletions

diff --git a/include/nn.h b/include/nn.h
index 871bd0f..3a2277a 100644
--- a/include/nn.h
+++ b/include/nn.h
@@ -31,7 +31,7 @@
 
 // somewhat internal, important for generating training data
 void nn_cell_idxs(uint8_t *out, tak_state_p state);
-void nn_prepare_head(float out_head[3], tak_state_p state);
+void nn_prepare_head(float out_head[2], tak_state_p state);
 
 void nn_prepare_input(float *cur_board, tak_state_p state,
                       const float embedding[NN_N_EMB]);
@@ -41,10 +41,10 @@ int nn_last_cell_idxs(uint8_t *out, tak_state_p state);
 // Network
 // ===================================================================
 
-float eval_net(const float *board, const unsigned int n_inp,
-               const float *w, const float *biases,
-               const float *logit_w, const float logit_b,
-               const unsigned int n_dense);
+float eval_net(const float *board, const unsigned int n_inp, const float *w,
+               const float *biases, const float *logit_w, const float logit_b,
+               const unsigned int n_dense,
+               const float parity);
 
 // Evaluators for the two built-in networks.
 float five_nn_fn(tak_state_p state);