Will computers dominate poker as they did with chess?

[Fractured_simile]

An A.I that could dominate a deepstacked high stakes cash game is a scary prospect- how much of the play at this level is dependant on your opponants state of mind and holding and other information such as table image, possibly false timing tells, specific previous hands the two of you have played etc., as a pose to your own cards and pot odds? i don't really do math so maybe 'game theory' is the magic bullet to couneract the intuitive elelments of the game, but i still think an A.I capable of modelling these things would probably dominate much more than poker. I still voted for eventually.

[pzhon]

[ QUOTE ]
The strongest Backgammon bots are neural nets that started off by playing many games against itself using no more than some dummy strategy, making silly mistakes, whilst learning from these (i.e. adopting its strategy towards those that would have yielded better past results). No human-strategies or other game-theoretical information gets loaded into these bots. Yet, after a few hundred thousand games, these bots can not be beaten by any human except for a few world-class professionals...

[/ QUOTE ]
You have confused separate events.

Tesauro created a backgammon bot which learned from self play and the rules of the game alone to play better than all other backgammon bots at the time (in the late '80s). This was a triumph of self-learning over older methods of trying to feed programs human expertise. However, this bot still played poorly by human standards.

Tesauro then created a strong bot which was close to the strength of the top players of the time. However, this was based on a better method of encoding the board, and other information from people.

The best backgammon programs now are much stronger than the ones created by Tesauro, and they rely more on human expertise. The top human players are close to or slightly worse than the top backgammon bots only because all top backgammon players train with bots.

By the way, the basic neural nets used by the top backgammon programs are not as strong as the top human players. However, computers are now fast enough to look ahead through the thousands of possibilities over the next couple of rolls, and this improves their evaluations substantially.

[curious123]

[ QUOTE ]
Bill Chen on crack can probably be capable of beating all the best players now.

[/ QUOTE ]

fyp

[jukofyork]

[ QUOTE ]
You have confused separate events.

[/ QUOTE ]
Sorry, but this isn't quite correct either. [img]/images/graemlins/smile.gif[/img]

[ QUOTE ]
Tesauro created a backgammon bot which learned from self play and the rules of the game alone to play better than all other backgammon bots at the time (in the late '80s). This was a triumph of self-learning over older methods of trying to feed programs human expertise. However, this bot still played poorly by human standards.

[/ QUOTE ]
Tesauro's first bot created in 1989 (called NEUROGAMMON) actually didn't learn from self-play - it's heuristic was trained using expert annotated training data
(see here).

[ QUOTE ]
Tesauro then created a strong bot which was close to the strength of the top players of the time. However, this was based on a better method of encoding the board, and other information from people.

[/ QUOTE ]
Tesauro's second bot created in 1995 (called TD-GAMMON) did use a method of reinforced learning called "Temporal Differencing" (see here) which was pioneered by Richard Sutton who extended ideas used by Arthur Samuel in 1959 to train his Checkers heuristic by self play alone (see here). Without going into detail; the basic idea of TD-learning is to minimize the error between consecutive evaulations rather than attempt to minimize the error between the current evaulation and the "reward" signal at the terminal state (ie: Win, Loss, Draw, etc).

If you are interested in reading more about reinforced learning or TD-learning then I suggest reading the relevant sections of "Artificial intelligence - a modern approach by Russell S. & Norvig P. (2nd Edition)" first, then moving onto "Reinforcement Learning: An Introduction by Sutton R. & Barto A." (see here for the full book in HTML format).

I also have a few links to other interesting resources related to TD-learning here, but it's been several years since I've updated any of those links so expect some to be dead...

Juk [img]/images/graemlins/smile.gif[/img]

[AaronBrown]

[ QUOTE ]
Aaron, whilst you have strong doubts whether a computer could ever survive playing against a table of top-rated human, would you accept that in a number of years a top-rated human no longer can survive against a table of (independent) bots?

[/ QUOTE ]
Actually, I suspect that might be true now, or would be if some top poker researchers devoted much effort to it. Anyway, I certainly agree that a table of computers will someday be able to bust a single human player, without collusion of any kind.

[jukofyork]

[ QUOTE ]
I don't think computers will ever be able to dominate a table of human players at no-limit. That's a different problem.

[/ QUOTE ]
[ QUOTE ]
Anyway, I certainly agree that a table of computers will someday be able to bust a single human player, without collusion of any kind.

[/ QUOTE ]
Could you explain your reasoning behind these two statements.

When you use the term 'dominate' in the first statement are you simply stating the computer player will:

a) Not have a significant advantage, but will have some advantage.
b) He will not be able to 'dominate' all the other players because they may occasionally use a set of (non-colluding) strategies that make it impossible for the computer to win?

I accept (b) is possible, but I don't really see where it relates to a question on the future of poker AI. This problem could just as easily be faced by any player (human or computer) with any configuration of human and computer players at the table.

I think rather than "Will computers dominate poker as they did with chess?" a better question may have been "Will computers eventually be at least as good as the very best players?" or phrased differently "Do you think that computers will always have some inherent disadvantage over humans when it comes to poker?".

Juk [img]/images/graemlins/smile.gif[/img]

[jogsxyz]

[ QUOTE ]


The main stumbling blocks along the way are:

a) Lack computing power.
b) Lack of applicable algorithms.


[/ QUOTE ]

That's a problem with programming chess and go. The approach to solving poker will be entirely different. Chess
and go can't be solved by monte carlo. Poker can.
A pokerstove type routine would be part of the poker
program. One million iterations. The poker program
would have access to opening hand charts and knowledge
of game theory.
Programming poker will be easier than chess or go.

[Pirron]

[ QUOTE ]
Bill Chen with a team of crack programmers can probably be capable of beating all the best players now.

[/ QUOTE ]

I was expecting this kinda of answer. The idea of applying only mathematics to make most optimal play in any poker situation seems to give a clear answer to the main question of this thread. I didnt read "The mathematics of poker" yet but based on the reviews of the book it seems possible to make a best play using only math.

The following question is how long it takes for programmers to create this perfect artificial player and what consequences you guys think it will bring for on-line poker?

[govman6767]

I would destroy skynet in a heads up match.

[RedManPlus]

[ QUOTE ]
What consequences you guys think it will bring for on-line poker?

[/ QUOTE ]

Predictions of "Bot Armageddon" are naive... in context of casino history.

Casinos ONLY care about the comfort and continuous play of average customers.

Both Poker Pros and Bots have an identical effect on Customers:

(a) scare the sh*t out of them
(b) impose an additional "rake"

Both Poker Pros and Bots have an identical effect on the Casino:

(a) transfer wealth from the Casino
(b) scare off the Customers

So distinctions between Pros and Bots are moot. Soon a major site will announce "shark-free" waters... where typical Customers can play without being skinned by Dark Forces. It only requires a simple rule: you screen out 2-3% of your most active accounts that are showing a profit... and kindly LIMIT those accounts to, say, 2 hours/day.

No one will care except the 2+2 community. 95% of Customers will celebrate. Profits will increase.