Version 0.10 released - Next steps
Version 0.10 is released now. If no major bugs surface in the next few days the server will start enforcing this version. There is this 1500+ post issue where most plans for the future were posted in the past. It's become rather problematic to read, especially on mobile, and mixed with a lot of theories (most not backed with any data or experiments 😉) so I'll post my plans and thoughts for the near future in this issue. It looks like we're slowly reaching the maximum what 64x5 is capable of. I will let this run until about 2/3 of the training window is from the same network without improvement, and then drop the learning rate. I expect that's the last time we can do that and (maybe!) see some improvement. I have been training a 128x6 network starting from the bug-fixed data (i.e. starting around eebb910d) and gradually moving it up to present day. Once 64x5 has completely stalled, I will see if I can get the 128x6 to beat it. If that works out, we can just continue from there and effectively skip the first 6500 Elo and see how much higher we can get (and perhaps do the same with even bigger networks) from continuing the current run. If that kind of bootstrapping turns out not to work, I'd be interested in doing a new run. My ideas for that right now: - Somewhere between 128x6 and 128x10 sized network. 128x10 would be 8 times slower, but there is a ~2x speed improvement that we could expect to have merged in by then, and the total running time would be around half a year
Posted by gcp on Jan 7, 2018, 8:20 PM
Tags
Related issues
Raise real-world problems, find people facing the same issue, and discuss solutions.4,500 issues0 comments
github (EdTech)
Message limit 10000 team edition
[NEW] Data Tiering - Valkey as general data store not just in-memory data store
Info: No releases on google play for now
proposal: spec: sum types based on general interfaces
Users can't use IdPs outside of the ones enumerated by RPs
CLIP Training Code
Implement a Set primitive to complement Dictionary and Array
better documentation of reborrowing
Version 0.10 released - Next steps
Version 0.10 is released now. If no major bugs surface in the next few days the server will start enforcing this version. There is this 1500+ post issue where most plans for the future were posted in the past. It's become rather problematic to read, especially on mobile, and mixed with a lot of theories (most not backed with any data or experiments 😉) so I'll post my plans and thoughts for the near future in this issue. It looks like we're slowly reaching the maximum what 64x5 is capable of. I will let this run until about 2/3 of the training window is from the same network without improvement, and then drop the learning rate. I expect that's the last time we can do that and (maybe!) see some improvement. I have been training a 128x6 network starting from the bug-fixed data (i.e. starting around eebb910d) and gradually moving it up to present day. Once 64x5 has completely stalled, I will see if I can get the 128x6 to beat it. If that works out, we can just continue from there and effectively skip the first 6500 Elo and see how much higher we can get (and perhaps do the same with even bigger networks) from continuing the current run. If that kind of bootstrapping turns out not to work, I'd be interested in doing a new run. My ideas for that right now: - Somewhere between 128x6 and 128x10 sized network. 128x10 would be 8 times slower, but there is a ~2x speed improvement that we could expect to have merged in by then, and the total running time would be around half a year
Comments (0)
No comments yet. Start the discussion.