Even though the link I posted is from Gote's (my) perspective, the embed is still from Sente's perspective.
Wow yeah that's weird, it didn't even try to stop you lol
Is it doing this every game? Or just the one?
wait this looks so weird to my eyes
hmm that's strange, I tried the exact same thing against lvl 3 CPU and it didn't work for me
lishogi.org/bdt2fyj6HpIG
Well mine was very weak too
Or I played so long not agaisnt the computer that i did not notice the change to fairy
lishogi.org/1QhRRLT4TEjx
Yep Level 4 was weak too
But level 5 with Yanerou was normal so i think fairy is the problem
lishogi.org/zoguY8jb
Impaße, Gote is victorious.
Yeah, it seems the Fairy Stockfish levels are all bad, which is odd, because Fairy Stockfish is a pretty good computer. Something's wrong with the leveling system for that specific engine.
Anyway, if all the lower levels mean is a lower depth, that is a bad system. I remember the level 2 computer thinking only one or two moves ahead, and even back when it was YaneuraOu (the non-buggy one), it was still a pushover. A node limit would likely be better, but we run into a different problem (which is also present in depth limit play): the computer's evaluation function is still too good. Either the engine entirely overlooks something and blunders, or it doesn't and plays the best move.
How does Lichess do it? It's likely we can just copy their homework.
I remember a while back, there was quite the issue with the CPU, one particular level was pushover easy, and the next level up crushed all beginners. It seems to be an ongoing issue even now, trying to get the play levels correct. I think one of the big issues is definitely the lack of options as far as open-source engines go. There's YaneuraOu, Fairy-Stockfish, and I think Dolphin?
Maybe it would be possible to get the partially-trained weights of YaneuraOU and use that for the lower levels? They released them a while back, but I can't tell if they are still being hosted:
yaneuraou.yaneu.com/2016/07/22/%e3%82%84%e3%81%ad%e3%81%86%e3%82%89%e7%8e%8b%e3%81%a7%e4%bd%bf%e3%81%88%e3%82%8b%e8%a9%95%e4%be%a1%e9%96%a2%e6%95%b0%e3%83%95%e3%82%a1%e3%82%a4%e3%83%ab28%e3%83%90%e3%83%aa%e3%82%a8%e3%83%bc%e3%82%b7/