"Poker" AI competition

Does anyone remember the Connect Four AI competition I held.. umm.. more than 5 years ago?

http://cboard.cprogramming.com/conte...t-results.html

Anyway, I remembered that competition today and I thought it would be fun to hold another. This time the game is Poker... or a simplified version of it.

Rules

1. Each player is dealt a hand, which is a uniformly random real number between 0 and 1.
2. The highest number wins
3. The first player has to bet 1, which is the big blind.
4. The players then makes their decisions: check, fold or raise
5. When every player has either checked or folded, the player with the highest number takes the pot
6. Regular Hold'em rules apply for all-in
7. The data type of money is integer.

Code
To enter the contest, implement the following function:

Code:

---

int make_bet(int my_id, float my_hand, const GameInfo&  game_info)
{
  return INT_MAX; //Always all-in
}

---

Where the struct GameInfo looks like this:

Code:

---

struct GameInfo
{
int num_players;

int* money; //Chip count for each player (including myself)
bool* has_folded; //What players are still in play
int* has_bet; //What each player has betted
int curr_max_bet; //Current highest bet
};

---

To check, the function make_bet must return (curr_max_bet - has_bet[my_id]). A smaller bet will be converted into a fold. A bet higher than money[my_id] will be converted to all-in.

You may use static variables to allocate lots of memory to keep track of your opponents previous moves. This page will probably be edited as people as questions etc.

Please write in this thread if you plan to enter, so that others know whether this is worth spending time on. The deadline for submissions is April 1st by PM.

My first impression of your game was "Oh, this is neat, he simplified poker so we can focus our AI on the betting". But with only one round of betting and no hand information, I don't really see it as poker. Meh, still interested, but I object a bit to calling it a "Poker" AI competition.

I have a question about the return value. You list "check, fold or raise" as the possibilities. Since there is only one round of betting and someone has to post a blind, the options would actually be "call, fold or raise".

You list

(has_betted[my_id] - curr_max_bet)

as the way to "check" (I'll assume you mean call). That would produce a negative number though unless you are the high better (such as when it gets back around to the big blind) in which case it would produce 0.

(curr_max_bet - has_betted[my_id])

sounds like what you probably meant, though it's late and perhaps I need to re-read?

"...const GameInfo& game_info)" so C++ only, right? And "has_betted" is awkward since "betted" isn't the proper past tense. Probably should be "has_bet".

How much money will each player start with? I would assume they would all start with the same amount, like a tournament.

Can we be guaranteed that num_players will remain constant throughout the entire run? If not then we can not keep track of other players reliably. If there are 6 players and player 1 is knocked out, then if num_players is reduced to 5 with player 1 being completely removed, all of our records become invalid.

How many players will be "at a table" at one time? I doubt all THAT many people are going to participate, so it's not like we'd end up with 140 AI players at the same table... but should we expect possibly more than 10 players to be "seated together"? Most I've ever played with is 11 at a table, though for Texas Hold'em it is theoretically possible to play with 22 players at a table (44 cards [22 players] + 1burn + 3 Flop + 1 burn + 1 Turn + 1 burn + 1 River), 23 if the dealer doesn't burn cards...

I like the simplicity of the design, but that simplicity also makes it impossible, from what I can tell, to get any information other than what players have bet and who won the hand. It's going to be more of a guessing game than anything.

Sounds good. I'm in.

Well, I tried.

Quote:

---

Originally Posted by jdragyn

(has_betted[my_id] - curr_max_bet)

as the way to "check" (I'll assume you mean call). That would produce a negative number though unless you are the high better (such as when it gets back around to the big blind) in which case it would produce 0.

(curr_max_bet - has_betted[my_id])

sounds like what you probably meant, though it's late and perhaps I need to re-read?

---

You are right. It will corrected.

Will there be a test harness provided ahead of time?

I agree with jdragyn, using a float in stead of a real hand removes the interesting features of poker. It makes a lot of intelligent decisions completely worthless. Also, never knowing what the other players had makes it completely worthless.
Why not make a real poker AI? It's not that difficult to code the framework.

Since very little knowledge of poker is required here, can someone explain this part?

6) Regular Hold'em rules apply for all-in

Quote:

---

Originally Posted by MK27

Since very little knowledge of poker is required here, can someone explain this part?

6) Regular Hold'em rules apply for all-in

---

I don't know what "regular" Hold'em rules are... but I'd imagine he's trying to say "No Limit" (even though in casinos a limit is more common).

That is to say at any point you can bet your entire stack of chips or up to whatever the second chip leader has in their stack (obviously you couldn't bet more than anyone else has)

I agree with the others, a simple Texas Hold'em simulation would not be that hard to write and it would make for a much more interesting competition.

A full simulation isn't necessary. I think this one is interesting enough.

A couple of questions:

1. Quote:

   ---

   Originally Posted by jdragyn

   Can we be guaranteed that num_players will remain constant throughout the entire run?

   ---

2. What languages are acceptable?

>> Why not make a real poker AI? It's not that difficult to code the framework.
>> a simple Texas Hold'em simulation would not be that hard to write
Yes, it would be pretty difficult.

I'd love to try Texas Hold 'em if either of you want to write the simulator and start another contest, but I fear even the AI for the player would be too time consuming for this format.

The simplified version presented here provides the problem of how to bet. Adding in full hold 'em rules just adds the complexity of calculating odds to make a better hand, but with this version you still get to keep track of player tendencies and betting behavior.

>> Also, never knowing what the other players had makes it completely worthless.
I don't think it's worthless, but I do think providing information on who wins each round and all the values for all exposed hands would be beneficial.

Quote:

---

Originally Posted by Daved

>> Why not make a real poker AI? It's not that difficult to code the framework.
>> a simple Texas Hold'em simulation would not be that hard to write
Yes, it would be pretty difficult.

I'd love to try Texas Hold 'em if either of you want to write the simulator and start another contest, but I fear even the AI for the player would be too time consuming for this format.

The simplified version presented here provides the problem of how to bet. Adding in full hold 'em rules just adds the complexity of calculating odds to make a better hand, but with this version you still get to keep track of player tendencies and betting behavior.

>> Also, never knowing what the other players had makes it completely worthless.
I don't think it's worthless, but I do think providing information on who wins each round and all the values for all exposed hands would be beneficial.

---

Well, the framework itself wouldn't be difficult. An actual AI might, as you may have to analyse the chances your hand will win, which is quite difficult. But you can have lesser AI's as well.
Why I think it's worthless that you don't get the other player's hands is that an AI might study the bet heights of the other players and compare them to how good the hand was. If this can ever be known, the AI would have to become more complicated.

Right now, all you have to do is calculate the chance your hand will win (hand_quality ^ (num_players - 1)), and simply study the bets of the other players to see if they are too high.
Really, you can't even figure out if you bluffed out an opponent or the opponent called but had a worse hand.

I agree that some additional knowledge of how the hand ends would be necessary, maybe Sang-drax can add that (remember that he asks for improvements and suggestions, this isn't set in stone).

At this point I'm the only one committed to playing, though. I doubt that he'll even bother if nobody else jumps in.

No you aren't the only one playing. I already sent sang-drax my file.

Oh, sorry. I didn't realize that's what your post meant.

I guess that means there's two of us.

Quote:

---

Originally Posted by EVOEx

I agree with jdragyn, using a float in stead of a real hand removes the interesting features of poker. It makes a lot of intelligent decisions completely worthless. Also, never knowing what the other players had makes it completely worthless.
Why not make a real poker AI? It's not that difficult to code the framework.

---

Actually, I quite like the idea of using a float to represent the "hand". It quantifies the hand in a simple way so the focus can shift to betting, which in my opinion is what poker is really about. You can't play poker without some form of wagering, even if it is pennies, poker chips or pretzels.

My primary reservation for the given format is the lack of information.

• If I fold, will my AI function still be called, even though game_info.has_folded[my_id] is true? If not, I will lose all further information for the hand.
• Even if I stay in to the end, if I am the first to raise and some players call, others fold, but no one raises, will it ever come back to me so I can poll the struct to see who stayed and who folded? If not I have no way to tell what the effects of my action were.
• If I call someone's big bet, I need to see what they had. Was their hand (number) really bigger than mine or were they bluffing? Or did they just overvalue their hand?

My secondary reservation, which is not quite as much of a problem, is the format.

• Except for some really old variations of poker (which I wasn't aware of until I did some wiki research), a hand of poker contains one or more changes, each accompanied by a round of betting.
  
  • Hold'em and Omaha have pre-flop, flop, turn and river
  • 7-card stud and Razz have 5 "streets", some face up, some face down
  • Draw has one or more draws before showdown (triple draw has 3 draws for example).
• These changes allow the players to gather more information, both during the hand and over the course of several hands. They also allow more complex patterns of betting.

I say the format is not quite as much of a problem because even a simple version of Draw poker allows a large amount of variation in play. Such complexity might be beyond the scope of a module crafted in only a couple weeks. If the format were to be expanded to include some change during a hand, the value of a hand could easily be kept as a float:

Code:

---

"Deal" the starting hand (random value between 0 and 1 for each player)
Blind betting round (big blind forced bet and responses from everyone)
Emulate a "flop" or a "draw" by adding a random value between 0 and 1 to each player's existing hand
Normal betting round (players may check)
Showdown - highest hand (which is now between 0 and 2) wins

---

I would like to submit an entry even if things stay as-is, though I would like the questions in my other post answered so I know what assumptions I can make from hand to hand.

Quote:

---

Originally Posted by MK27

Since very little knowledge of poker is required here, can someone explain this part?

6) Regular Hold'em rules apply for all-in

---

Given:

• Player A has $20
• Player B has $50
• Player C has $75
• Player A goes all-in (bets his entire $20)

Scenario 1: Players B and C can simply call. The pot now has $60 ($20 from each player). Whoever has the best hand wins the entire pot.

Scenario 2: Player B raises to $50 and Player C calls. $20 of Player B's bet goes to the main pot to match Player A's $20, and the other $30 is placed in a side pot. Player C calls, so $20 more goes into the main pot for a total of $60 and $30 more goes into the side pot to match B's $30. First the side pot goes to B or C, whichever has the better hand, because only B and C contributed to that pot. Next, the main pot goes to the better hand of A, B or C because all 3 contributed to the main pot.

Scenario 3: Player B raises to $50 and Player C re-raises to $75. Exactly like Scenario 2, except the extra $25 that Player C raised would go into a second side-pot. If there are only these three players, Player C gets that $25 back no matter what because only C contributed to the second side pot (he most definitely has the best hand amongst himself).

Scenario 3a: If we add Player D with $100 and he calls the $75, the main pot now has $80 ($20 from each), the first side pot has $90 ($30 each from B, C and D), and the second side pot has $50 ($25 each from C and D). The second side pot of $50 would go to the better hand of C or D because they contributed to it. The first side pot of $90 would then go to B, C or D because they contributed to it. And then the main pot of $80 would go to A, B, C or D.

Quote:

---

Originally Posted by SlyMaelstrom

I don't know what "regular" Hold'em rules are... but I'd imagine he's trying to say "No Limit" (even though in casinos a limit is more common).

---

Actually, all of the popular formats that I am aware of have the same type of All-in rules that I described above. The format Sang-drax is proposing would be considered No Limit, so a player can bet up to their entire stack on any betting round, vs. Limit where there is a limit to how much can be bet, and usually it is a fixed number.

A $5 limit game means if you want to bet, you must bet $5. If you want to raise, you can raise by $5. If it comes around to you to call, your call will be in an increment of $5. If you want to raise but only have $4 left, you would be considered all-in. Everyone who calls your raise would be calling $5, with $1 from each going to a side pot.

There is also Pot-Limit, which is similar to no-limit (you can bet any amount rather than in steps), but your bet can not be larger than what is currently in the pot (including bets from the current betting round). This betting variation usually limits the betting early in the hand and keeps things a little more conservative.

can i just have the exact (as far as they are currently agreed ) rules for this contest i would like to give it a go, people have already submitted entires so i think unless that one is withdrawn then we should all submit programs based on the criteria of that entry...?

Unfortunately Sang-drax hasn't been active here in two weeks, so who knows what his response will be to the suggestions.

I might just try to do a simple AI based on what we have here and then modify it if he updates anything. I've been waiting to hear back up until now.

I've completed an entry based on the original post information (despite all my suggestions and comments...), just want to wait and see if Sang-drax makes any changes before I send it in.

I'm still stuck trying to get the test harness working correctly. I have to come up with a way to handle side pots, not doing so was screwing up my test runs.

Quote:

---

Originally Posted by Daved

Unfortunately Sang-drax hasn't been active here in two weeks, so who knows what his response will be to the suggestions.

---

Yes, I apologize for that. I have been traveling.

It is correct that I meant no-limit hold'em. Hopefully that was clear since all-in was mentioned. There is no pot-limit.

As pointed out, there is no interface for actually knowing how the hands end. If that is desired, I could add an interface for that. I tried to keep things simple, in order to encourage people to actually submit something. How would you like the interface?

Business travel or personal? Either way, hope your travels were interesting. :)

The interface doesn't really need to change. So long as we have access to the struct, and it is updated appropriately, we can deduce any information we need. I kind of see that as part of the challenge now.

I do have a couple suggestions if you want to go ahead and provide some additional info at the end of a hand:

Add a bool member to GameInfo, called end_of_hand. If this value is false, the framework is expecting a bet. If this value is true, the framework will ignore the return value; it is simply exposing the struct for us to update after the hand. Alternatively, pass the hand value as a negative value to indicate the end of hand update call.

Add a float member to GameInfo, called high_hand. If end_of_hand is true, then high_hand will expose the hand value that won the hand or ALL_FOLDED to indicate the hand winner didn't have to show because everyone else folded. If end_of_hand is false then high_hand will be IN_PROGRESS or something.

Add an int member to GameInfo, called hand_winner. If end_of_hand is true then hand_winner will expose the player id of which player had the high_hand. Otherwise it could be NO_WINNER.

An easy way to implement this is, after the betting is complete, determine the winner, update the GameInfo struct with the hand_winner and high_hand values, set end_of_hand = true and then call each AI regardless of whether they folded or showed-down.

The only problem with this method would be if there are side-pots and/or split pots. In that case though, high_hand could be a pointer. high_hand[0] would be the hand that won the main pot; high_hand[1] would be the hand that won the first side pot, etc. Likewise , hand_winner could also be a pointer with winner of the main pot being hand_winner[0] and so on. The only addition this would require would be a count of how many pots were formed during the hand. This will add a bit of complexity to the framework, but it would allow our AI to capture more complete info about a hand if it so chooses.

Or just leave it as-is and we'll see who can write a "better" AI with the limited info. I'll be happy either way.

>> Yes, I apologize for that. I have been traveling.
No worries. I'm just glad that you're running the contest and there are a few other entries.

If we only have three entries, are we going to play three handed?

Quote:

---

Originally Posted by jdragyn

Business travel or personal? Either way, hope your travels were interesting. :)

---

Conference, which was interesting. :)

Quote:

---

Originally Posted by jdragyn

I do have a couple suggestions if you want to go ahead and provide some additional info at the end of a hand:

---

Those are really good suggestions, but as you point out, it gets complicated with the side pots. Maybe knowing the winner of the main pot would be enough?

I think that unless you wanted to extend the deadline, leaving it as-is is probably going to be best.

My entry has been PM'd. It's pretty simple (no, it's not just a "return INT_MAX;"). Good luck Daved and whiteflags :)

I finally had time to get a (somewhat) working test harness... so I guess I have only a couple days left to make a good algorithm. I'm guessing mine will be pretty simple, too.

Ok, I submitted mine.

I got busy with work so I didn't have time to add some more complex ideas I had. I guess we'll see what happens.

OK, so I have three submissions: whiteflags, jdragyn and Daved.

Now it's Easter and I am going away. But I'll bring my laptop and hopefully the results won't take that long. This is going to be interesting.

Enjoy your Easter! Unless you really wanna geek out for your own sake and get the results, I am in no hurry and will be happy to wait until next week. Cheers!

I'm on the train now and have coded the framework. Daved seems to be doing well. I still have to make sure there are no bugs before I post any results.

Results here: http://cboard.cprogramming.com/conte...ai-results.htm