handicap games between top human players and engines

Have there been any shogi games on record in the last 3 years, either in person or online, in which a top shogi engine has given bishop or rook handicap to a top level human player (either a pro or a very top amateur)? My guess is that the software can give a rook to top amateurs and to midrange pros, and a bishop to the very top pros, but I haven't heard of any such games being played, although they are reasonably common in both Go and chess. In Go I'm told that the top pro needs two stones while ordinary pros need 3, which would correspond to what I say above in shogi. In chess the top pros need two pawns, while ordinary pros need knight odds.

I'm not aware of any recent games between pro humans and top engines.
But I've simulated such games by letting fully strength YaneuraOu-Suisho5 play games against Lishogi AI level 8.
Here is a small collection: lishogi.org/study/N0HThRjS (note that the automatic analysis isn't as strong as the strongest player in these games, so it's kinda useless, but gives an approximate idea of when a game is lost.)

4-piece odds seems to give YaneuraOu-Suisho5 a good challenge.
6-piece odds is only possible with a good amount of luck, same for Rook/Bishop gained.

Shogi, Crazyhouse (www.pychess.org/HckZWrgA) and other games with drop/capture to hand rules allow for significantly higher odds that Chess, knight odds Stockfish vs Stockfish lvl 8 is a loss most of the time (thought not always: lichess.org/0NTNjoKd, even queen odds works out once in a blue moon: lichess.org/hSMsZFPD).

Hardware matters a lot too, in Go managed 3 stones with 5.5 komi (online-go.com/game/51638842) against Katago-micro [9d] on a 5 year old consumer GPU, but couldn't win 3 stones with 0.5 komi. A sponsored event with multiple professional grade GPU's/TPU's would be a different matter though. Especially if the net was specifically made for handicap games.

I'm a bit shocked by what you write; Lishogi AI level 8 in the four piece handicap game also used YaneuraOU it seems, just a slightly older/weaker version I think. Isn't level 8 full-strength version, just weaker due to difference in hardware? If so that would hardly seem like enough reason for such an enormous handicap. Or is it weakened/limited in some other way besides hardware? What was the time-limit? This is very strange because I can generally beat YaneuraOU at two piece handicap, and I am perhaps somewhere between Level 5 and Level 6 on Lishogi AI; I can't beat level 8 even with rook and lance handicap. So it seems this tells us nothing about how top humans would do against best AI, they would obviously need less than two piece, but how much less is very hard to guess. I'll bet there have been a lot of such handicap games played by pros, but perhaps just privately to avoid embarrassment or criticism. Perhaps there are at least reports of such private matches, even if the games are not public?

AI level 8 isn't the strongest version, it's indeed a weakened setting, for playing against humans.
The in-browser analysis board is the full strength version of the Web Assembly implementation of Yaneura.
But Yaneura running natively instead of in-browser is much stronger still and allows for the use of the newest, strongest nets.

AI level 8 isn't very goods at simulating humans though and doesn't understand that it's facing a much stronger opponent.
So unlike pro human players it doesn't play extra carefully and doesn't know anti-engine strategies.

In these games there is no real time limit, AI level 8 never takes more than a few seconds, YaneuraOu-Suisho5 is given about 10 to 40 minutes per moves (compensating a bit for the fact it's not playing on a super computer). In the six piece game the opponent isn't Yaneura level 8 but Fairy-Stockfish level 8, it's much weaker.

I agree it's not a fair comparison to humans, humans have a different kind of understanding of the game, that would probably allow them to win the four piece games.

On the other hand, it may just take a single mistake from a human to get steered away for their safely prepared anti-engine strategy into a tactical line that eventually becomes so complex that no human could oversee it.

You're probably right about the private matches, pros don't really have anything to win from public handicap games I think. But I'd love to see some, I enjoyed Hikaru's two pawn odds chess games against Komodo and his earlier games against Houdini.

So part of the explanation is that Suisho5 was taking more than 100 times as much time per move on average; my experience with it is having it set to 5 seconds per move on powerful hardware.

In chess, with Stockfish level 8 on Lichess I know that it is set to play full strength subject only to depth and time limits; unlike lower levels it never deliberately plays an inferior move. I assumed that this would be the same with Shogi AI level 8, but I don't know this. When you say it is "a weakened setting", do you mean weakened only by depth/time limits, or weakened in the sense of the lower levels where it will deliberately play inferior moves sometimes?

Is there any data on how the Lichess shogi AI level 8 (with Yaneura) performs in even games against the best humans? I'm sure the top pros do play against it, but perhaps the games and/or results may not be public?

It is true that the pros wouldn't have much incentive to take a handicap in public games. But this seems to be common in Go, which has a similar culture to Shogi, so I find that a bit puzzling.

By the way, Hikaru's chess games at two pawn odds were not against a browser version of Komodo Dragon; it was playing on a very powerful 32 core computer.

I generally find that with two piece handicap against Suisho5, it takes many mistakes to lose. Normally I am totally winning, it embarks on some clearly desperate attack, and I only lose if I badly misplay the endgame. But at rook and lance, I often lose badly, and only very rarely actually win a game. This is a huge step in the handicap series.

I'm not 100% sure about how Lishogi AI level 8 is set to play, but someone here probably knows: github.com/WandererXII/lishogi/discussions

Historically most famous human vs computer matches have been played against super computers (like Kasparov vs Deep Blue and Lee Sedol vs AlphaGo), that's why I chose the ~100x time odds on normal hardware.
(btw I fixed the part about Komodo).

It's true that it take many small mistakes to lose, but a strong engine that looks far enough ahead can steer a human player into very tricky positions here bigger mistakes are inevitable.

Probably AI level 8 is only depth/time limited, as I never see it make an obvious error. I note that it is estimated to be rated 2600, whereas the top human player (online play, not correspondence) seems to be just 2143 (in Rapid, lower at other speeds). But I have no idea how strong that player is. Does anyone know whether the best online player in lishogi is actually a top Japanese amateur, say 6 Dan or better (or even a pro), or just a reasonably strong amateur? I'm trying to get a sense for whether level 8 is below, at, or above pro level.

The difference between a "supercomputer" and a very good PC that one can buy for say $10,000 or less is not that much anymore for chess or shogi. At least with NNUE, there isn't a huge benefit to say 1000 CPUs vs say 32 CPUs, it's not that easy to use them. Maybe the difference is greater for engines that use the GPU.

Suisho5 can definitely steer the game into tricky positions as you say at rook and lance or smaller handicaps, but with two piece handicap or more, when it has no long range pieces (except the "crippled" lance) it can't readily achieve this. Whether it has a major piece or not makes all the difference.

This topic has been archived and can no longer be replied to.

Reconnecting