Re: Live Updates of the Laak vs Computer Match HERE
 

[ QUOTE ]
furthermore, NL HU poker has been SOLVED for ~12 BB or less.

[/ QUOTE ]

Of course it has been solved. It was solved a long time ago actually. Royal flush beats a straight flush, which beats four of a kind, which beats a full house, which beats a flush, which beats a straight, which beats three of a kind, which beats two pair, which beats one pair, which beats no pair high card.

All of these obviously beat David Benyamine.

Re: Live Updates of the Laak vs Computer Match HERE
 

[ QUOTE ]
[ QUOTE ]
July 24th - 3:38 PM PST
Phil +$1290 (488 hands played)
Big Hand. Hand 488 sees Laak dealt pocket Kc Kd. Phil raises - Polaris calls. Flop comes Qh 2h 4d. Phil raises again, and Polaris just calls. Ac hits on the turn and Phil pauses. Tilly chimes in from the crowd saying Polaris may know the ace is a scare card. Phil folds his Kings and and his lead is down to $1290.


[/ QUOTE ]
I wonder if this is accurately reported. I mean, really?

[/ QUOTE ]

Maybe he folded to a some kind of turn raise, but still, WTF

Re: Live Updates of the Laak vs Computer Match HERE
 

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
[ QUOTE ]


the computer and the human team plays each set of cards against the same set of cards. So one side cant get a good or bad run of cards.

[/ QUOTE ]

Yes, but for instance, Phil Laak suddenly decided on Hand 250 that he was going to be aggressive for awhile. Presumably Ali is doing his own thing. What if Phil's side gets a good or bad run of cards at that very moment?

Since they have studied this, presumably U. of A. has some reasoning as to why this will not affect things.

[/ QUOTE ]

i dont think you can think of it like that. I wish that the two people were different (like HU limit specialist), so we could assume that the team would be closer in sync, but testing wise, i think this gives a pretty accurate picture

[/ QUOTE ]

i dont know much about lack other than that everyone thinks hes a tourney donk. ali beats super high live games and deathdonkey said hes a great player. thats good enough for me.

[/ QUOTE ]

In a normal HU LHE match the standard deviation is about 4-6 sbets/hand (but it can be higher if both players are very aggressive).

If you play in duplicate pairs, ISTR it drops to about 2, meaning that if you play 50 hands, then replay then with hole cards reversed you get a std.dev. of about 20 sbets over the 100 hand match.

Over a 2000 hand match, 1 standard deviation in your bankroll is then 2*sqrt(2000) = 89.4 so if the humans were to break even over the next two sessions, they would lose by about 100 small bets, which wouldn't be significant. ('significant' is generally taken to mean at least 2 standard deviations.)

Marv

Re: Live Updates of the Laak vs Computer Match HERE
 

Score one for the humans!

Re: Live Updates of the Laak vs Computer Match HERE
 

I like Phil's strategy of switching into hyper-aggressive mode for short stretches, then tightening up when he thinks the AI has adjusted to him. Then later giving it another dose of hyper-aggressive to mess with it again. Even if the program is very solid from a game theory standpoint, the opponent modelling part of the program might be something the humans can manipulate and exploit, by playing against type.

Re: Live Updates of the Laak vs Computer Match HERE
 

[ QUOTE ]
('significant' is generally taken to mean at least 2 standard deviations.)

[/ QUOTE ]

Didn't know there was this general understanding. I guess that explains this otherwise meaningless blog entry:

[ QUOTE ]
4:13pm The final match results are in. The humans did win by a margin of $820 which is statistically significant.

[/ QUOTE ]

Re: Second Session Just Started
 

For those curious, Poker Academy put some graphs up and a couple of hands from each session: Match Graphs and Hands

Re: Second Session Just Started
 

2+2 should really dissect these research papers that the U of A is putting out.

http://www.cs.ualberta.ca/~games/poker/ (scroll down for publications)

i have read most of Morgan Kan's thesis and there is some REALLY interesting stuff. I would love to have a calculator that could calculate stuff like DIVAT, EHR, IHR, 7cHR, it doesn't look like it would be hard to make. it would definitely be nice to have in addition to pokerstove calculations.

it would really help with analysis around here i think, particularly with short handed and heads up hands.

Re: Second Session Just Started
 

[ QUOTE ]
For those curious, Poker Academy put some graphs up and a couple of hands from each session: Match Graphs and Hands

[/ QUOTE ]

I just noticed that you are one of the computer scientists at U of A.

any chance of you writing a bit for 2+2 about Polaris or just heads up strategy in general (maybe a quick run-down of your thesis?)? the 2+2 forums do a lot of work on poker strategy but it is largely unscientific, it would be nice to have a researcher post something about optimal heads up poker strategy.

Re: Second Session Just Started
 

[ QUOTE ]
2+2 should really dissect these research papers that the U of A is putting out.

http://www.cs.ualberta.ca/~games/poker/ (scroll down for publications)

i have read most of Morgan Kan's thesis and there is some REALLY interesting stuff. I would love to have a calculator that could calculate stuff like DIVAT, EHR, IHR, 7cHR, it doesn't look like it would be hard to make. it would definitely be nice to have in addition to pokerstove calculations.

it would really help with analysis around here i think, particularly with short handed and heads up hands.

[/ QUOTE ]

The last three are pretty trivial, though 7cHR is worth precomputing as a big table for the flop (and maybe for the turn too depending on your application) 7cHR preflop you just precompte and store.

DIVAT is more complex since it relies on a database of fair values which takes some time to build. It also only applies when you have knowledge of both player's hole cards so its a bit limited for real poker games, but OK for the publicity stunts^H^H^H^H^H^H^H^H^H^H man-vs-machine matches when full information is available at the end of each session.

Marv

Re: Second Session Just Started
 

Marv,

i am not interested in any of the numbers for in game calculations, i would just like to evaluate my play after the fact, kind of like how people plug a hand history into pokerstove just to see where they were at throughout the hand. so needing complete info for divat is largely irrelevant.


i agree all those metrics outside of DIVAT should be easy to calculate, which is why i am hoping someone creates such a calculator because if i attempted it, it would be pretty clunky. i only have one semester of C++ under my belt from a CS course i took 5 years ago, stuff like this makes me wish i tacked on a CS major.

Re: Second Session Just Started
 

Sure,

I was a former student/researcher at the University of Alberta. Research there started with limit ring games and evolved into various flavors of Poki. These include formula-based versions and simulation-based versions.

To isolate some variables and focus more simply on the underlying problem of dealing with imperfect information. This research switched over to heads-up limit research which delays having to tackle the multiplayer complexity until later.

Research into the limit heads-up bots has primarily fallen among two main thrusts: solving/approximating best-response equilibrium play and exploitive play.

The two most widely known examples of these different lines of research are Sparbot for the equilibrium case and Vexbot for the exploitive case. I'm not going to get into a debate of their quality as that is best for scientific papers/contests/matches to assess but as technology they are now getting to be a few years old allowing many opportunities for improvement in the bots that are participating in this year's Man vs. Machine match and in the AAAI Poker Program Championship or just independently created and kept a secret.

The main idea behind the best-response equilibrium programs is that they can put a guarantee on worst case behavior. No matter what someone does there is only so much they can lose by or in a perfect world they could guarantee to not lose. They basically build their strategy around being an impenetrable fortress.

On the other hand, the idea with the exploitive bots is that they look through their opponent's tendencies and try to construct the good counter strategies. In essence, they risk being exploited in an attempt to exploit their opponent.

Aside from building the bots to play poker, there has been another additional line of research as you mentioned (in Morgan's and Darse's theses) which focuses on attempting to get a fairer assessment in poker quicker. This line of research attempts to lessen the noise of a match's results to try and isolate who was the likely winner despite the money line results.

That's a basic summary of the largest bodies of research that has been done to date. Recently, there has been additional interest in NL as it adds more variables into the mix, and since research in the area of poker has been increasing recently, I think it is a safe assumption that research will start to heads towards multiplayer and other poker games as well.

In terms of poker research, there will be lots of cool results coming out of AAAI this year so keep your ears to the ground on that front. Aside from this greatly entertaining Man vs. Machine championship, there was also a Poker Program Championship whose results should be made shortly since the conference is currently going on and it will be interesting to see how far the state of AI has advanced since last years AAAI results.

Sorry for the long post, but I tried to give you the cliff's notes as you requested, but there was quite a bit to breakdown and summarize.

Terence.

Re: Second Session Just Started
 

Marv was just dissecting the numbers for you like you requested. The last three numbers are ways of assessing the strength of a poker hands. DIVAT is a way of assessing match results but is based on perfect information hands (where all hole cards are known) so you need information about all hole cards to apply it which was his point about why it can be used in the Man vs Machine match. To work with hands with not all hole cards known, it would have to be extended which Marv pointed out. It currently is also only formulated for the heads-up case since that is the current case of interest and it would also have to be extended to the multiplayer case for use there.

Terence.

Re: Second Session Just Started
 

Terence awesome post. i hope you stick around here and post in the future. same goes for any other U of A guys lurking in this forum.

Re: Second Session Just Started
 

Marv and eastbay have some great posts here on 2+2 so if you are interested in the subject you might enjoy going through them as well.

Terence.

Re: Live Updates of the Laak vs Computer Match HERE
 

Just letting everyone who is interested know that the fourth and deciding final match is going on and is about a fifth of the way through. Everything was exactly knotted up going into this match with each team having 1 win, 1 loss, and 1 tie.

Re: Live Updates of the Laak vs Computer Match HERE
 

Humans win, as I thought they would. Congrats to University of Alberta team for doing some great work and making headway on a very difficult problem, even though a lot of people here seem to think it should be easy to write a good program.

Especially encouraging from the human side is that they did better on the second day of play, leading one to believe that they adapted to their unique opponent, as any poker player should when going against someone new.

Maybe this will put an end to the talk of "omg computers are so smart they can add numbers and stuff, it's easy to write a bot that can beat humans in anything all the time." At least it will stop that talk for 3 days maybe?

Re: Live Updates of the Laak vs Computer Match HERE
 

Yup, big congrats to Phil and Ali. Very nicely done.

Re: Live Updates of the Laak vs Computer Match HERE
 

[ QUOTE ]
Humans win, as I thought they would. Congrats to University of Alberta team for doing some great work.

Especially encouraging from the human side is that they did better on the second day of play, leading one to believe that they adapted to their unique opponent, as any poker player should when going against someone new.

Maybe this will put an end to the talk of "omg computers are so smart they can add numbers and stuff, it's easy to write a bot that can beat humans in anything all the time." At least it will stop that talk for 3 days maybe?

[/ QUOTE ]

for the record, there is no question that this bot would likely clean up on-line. i don't think a bot has to be the absolute best to be respectable. Laak and particularly Ali (based on reports of his play, i have no first hand knowledge) are almost certainly better than ~98% of serious players.

/end pro-computer spin

Re: Live Updates of the Laak vs Computer Match HERE
 

[ QUOTE ]

for the record, there is no question that this bot would likely clean up on-line. i don't think a bot has to be the absolute best to be respectable.

/end pro-computer spin

[/ QUOTE ]

I think it's very very premature to say this (well, depending on what level. I think they've been able to beat novices for awhile), but I am open to the possibility and will be very interested in the hand histories.

Edit: I would also love to see this program take on the better rules-based bots online that Mr. Gatorade has called out.

Re: Live Updates of the Laak vs Computer Match HERE
 

Hey guys,

I've been following the posts in this forum over the last couple days, and I'm glad the initial response to the humans victory tonight is not that "of course the humans won". To be clear, I think the humans really benefited from a lot of information revealed by the bot team. If this was a fully competitive match, the humans might've been in tough since they wouldn't have known that the bot that they played in the last match was an equilibrium non-learning bot.

I hope you all enjoyed my blogging, and I'm also rather flattered that people are looking at my thesis. If you have questions about DIVAT, feel free to ask.

Morgan

Re: Live Updates of the Laak vs Computer Match HERE
 

[ QUOTE ]
Hey guys,

I've been following the posts in this forum over the last couple days, and I'm glad the initial response to the humans victory tonight is not that "of course the humans won". To be clear, I think the humans really benefited from a lot of information revealed by the bot team. If this was a fully competitive match, the humans might've been in tough since they wouldn't have known that the bot that they played in the last match was an equilibrium non-learning bot.

I hope you all enjoyed my blogging, and I'm also rather flattered that people are looking at my thesis. If you have questions about DIVAT, feel free to ask.

Morgan

[/ QUOTE ]

Morgan,

Thanks for the great updates. I've been on these forums for a few years and the subject of bots come up a lot. I think the attitude tends to be the opposite of what you say. People tend to underestimate how tough the poker problem is for an AI. This frustrates me both as an AI grad student like yourself, and it also frustrates me as a poker player. I think I was pretty much the only person predicting a human win. [img]/images/graemlins/smile.gif[/img] If you would have won, everyone would have said, "Of course the bot won, I could write a bot too that would win if I had the time and inclination". And that is annoying.

A loss to 2 great players doesn't mean you guys didn't do great work. The competition is more of a referendum on the poker domain than on the particular bot. There are so many aspects to it, and just skimming the papers, it seems like there is a lot of interesting material. And of course the lessons learned in the poker domain will be of use to AI in general. Keep up the good work!

Edit: Oops, I meant UNDERestimate. Fixed.

Re: Live Updates of the Laak vs Computer Match HERE
 

Morgan,
do you plan on releasing any programs or source code relating to DIVAT? i know that this might not be the ideal application for it, but i could see myself using it on individual hands (or is the DIVAT number meaningless for such a small sample?) that went to showdown in heads up matches that i played. i think it could really give me a really good starting point for analysis, of course it would be way too burdensome to calculate by hand.

Re: Live Updates of the Laak vs Computer Match HERE
 

I'm up way too late here - just got back from hanging out with Phil and Ali after the match. Ali and his girlfriend Ana are both incredible people to talk to. They are smart, funny, and just awesome to hang out with. Believe me when I say, Ali really is an amazing poker player. He had the bot figured out in the last match, there is no doubt about that.

Someday, DIVAT will be available as a public service. It'll take some work, but someone will do it even if it isn't me. I have to figure out what exactly I'm doing with my life now that this competition is over. There are rumours that it'll happen again within a year. We'll see how it goes ... I might just stick around to be part of it again. Either way, I should really work on cleaning up the code and making it available either in web format or release some source. We'll see how it works out, but I certainly feel that the tool is powerful enough that it'd be worth the effort to make available through some means. I'll make sure to post to twoplustwo when/if this happens.

Morgan

Re: Live Updates of the Laak vs Computer Match HERE
 

Why publish the source code the bot rather than just the thesis/writeup regarding it? Just seems like the additional additional academic value gained is much less than the inevitable resultant abuse of the source code. Of course, not publishing it is likely just delaying the inevitable - but all the same.

Re: Live Updates of the Laak vs Computer Match HERE
 

NY Times reported on the match - http://www.nytimes.com/2007/07/26/business/26poker.html

Re: Live Updates of the Laak vs Computer Match HERE
 

[ QUOTE ]
Why publish the source code the bot rather than just the thesis/writeup regarding it? Just seems like the additional additional academic value gained is much less than the inevitable resultant abuse of the source code. Of course, not publishing it is likely just delaying the inevitable - but all the same.

[/ QUOTE ]

This is very true. Opening up the source would undoubtedly cause your bot to be playing on real money sites on the internet in a short period of time, which is bad for everybody.

Also, the code is (presumably) so specific that the open source magic of software evolution probably wont result in your software getting any better because everything is so proprietary.

Re: Live Updates of the Laak vs Computer Match HERE
 

To be clear I'm talking about getting DIVAT ready for release in some form. I'm not sure if it'd be source code. Maybe it'd be usable as a public service. I'm not too sure at this point, but it's something that should be done.