Abstract - University of Nottingham
University of Nottingham
Supervisor - Graham Kendall
gp-gxk2
2nd Year Software Engineering Project -
“Texas Hold ‘Em”
Group Members: wxa00u Wafa Al-Ajmi
wlc00u Wai Leuk Chu
kxg00u Keeran Gohil
mjn00u Micheal James Nolan
vxp00u Viral Patel
txw00u Timothy Wood
[pic]
[pic] [pic] [pic] [pic]
‘Final Group Report’
May 2002
Contents Page page 2
1. Abstract page 5
2. Introduction page 6
2.1. Initial Problem page 6
2.2. Brief Introduction page 6
3. Research page 8
3.1. Brief History of Poker page 8
3.2. Poker Variations page 8
3.3. Rules of the Game page 9
3.4. Betting Strategy page 9
3.5. AI Computer Modelling Methods page 9
3.6. GUI - Design Options page 10
4. The Back-End page 11
4.1. The Need for a Back-End page 11
4.2. Research page 11
4.3. An Example Back-End page 11
4.3.1. A working Model page 12
4.3.2. Perfect Information page 13
4.3.3. Obvious Limitations page 13
4.4. Developing the Model page 13
4.4.1 Stripping the Back-End page 13
4.4.2 Modifications to Accommodate a GUI page 14
4.4.2.1. Getting a Card page 14
4.4.2.2. Getting a Decision page15
4.4.3. Putting it all together – the Back-End Class page16
4.4.4. Putting it all together – the Poker Class page16
5. Building a Computer Poker Player page18
Poker Theory page19
Implementations page20
6. Front-End Design - GUI page21
Initial Ideas page21
Mock Up Design page21
7. Creating the GUI page22
7.1. Initial Design page22
8. Developing the GUI page24
8.1. Limitations of the Interim GUI page24
8.2. Developing a Front-End page24
8.2.1. GUI layout and Features page24
8.2.2. Interaction With The User page25
8.2.3. Establishing Initial Game Data page25
8.2.4. Representing Dealt Cards page26
8.2.5. Establishing User Decisions page27
8.2.6. Board and Hold Cards page 28
8.3. Implementation page28
8.3.1. Implementation details page28
8.3.2. Back-End & Front-End Communication page29
9. Back-End & Front-End Communication page30
9.1. Background to Message Passing page30
9.2. Research page30
9.3. Control page31
9.3.1. Back-End to Front-End Message Passing page31
9.3.2. An Intermediate Class page32
9.4. Message Area page33
9.4.1. Initial Requirements page33
9.4.2. Naïve Implementation page33
9.4.3. Producer-Consumer Analogy page34
9.4.4. Building the Message Area page35
9.4.5. The Implementation page36
9.4.6. The Messages page37
9.4.6.1. Message Syntax page37
9.4.6.2. Implemented Messages page37
9.4.6.2.1. Messages to the Front-End page37
9.4.6.2.2. Messages to the Back-End page38
10. Division of Labour page39
10.1. Meetings page39
10.2. Web Site page39
10.3. Time plan page39
11. Evaluation page40
11.1. Limitations of the Software page40
11.1.1. Limitations of the GUI page40
11.1.2. Limitations of the Message Area page40
11.1.3. Limitations of the AI page41
11.2. Overall Successes of the Project page42
12. Bibliography page43
Appendix page45
Appendix Ai page45
Appendix Aii page45
Appendix Bi page46
Appendix Bii page46
Appendix Biii page47
Appendix Biv page47
Appendix C page49
1. Abstract
Games are an interesting and challenging domain for Computer Science Research. However, developing a program to play a strategic game involves converting application of theoretical concepts to practical situations.
Recently Poker has received a lot of attention from Computer Scientists and game researchers. The game is logistically simple yet strategically complex. Most importantly, Poker is a non-deterministic game with imperfect information. Due to these factors Poker provides an excellent domain for investigating problems of decision making under conditions of uncertainty.
In this report we will discuss how we went about creating a high class automated Poker playing program that plays a legal game of Poker. Each section will describe how the separate points were addressed and how we sorted out any encountered problems. Also, some sections are divided into subsections giving detail of all the steps that were followed to create our high performance Poker player.
2. Introduction
Initial Problem
In this section of the report we will describe the initial task that was set for our Group project at the beginning of the academic year.
Texas Hold 'Em (gp-gxk2)
This year Dr Kendall is supervising two projects but both groups will be given the same project and will have to produce a piece of software that competes against the other group and, potentially, other players as well. The competition will take place at some time convenient to ourselves (possibly the open day). The group that wins this competition will be awarded a prize of £75.
Your task is to develop an automated poker player that plays "Texas Hold 'Em". This is the variation of poker used in the World Series of Poker played in Las Vegas each year.
At the end of the project I will arrange a competition so that the two groups can play against each other (and possibly other players) to see which group can win most (or lose less!!) chips. The group that wins this competition will be awarded a prize of £75.
The competition will take place at , on one of their Texas Hold 'Em play tables (so you should use this as a test bed). When we run the competition we will choose a table to play at (which could have other players as well), both players will start with the same number of chips (which I will announce just before we start) and we will play for a given time period (which I will announce just before we start). Whilst the competition is underway you are not allowed to change the way the automated player plays. You are only allowed to enter commands that keeps it informed of the game status and allows it to decide what to do.
As well as playing a legal (and hopefully good!!) game of poker your project should also be graphically pleasing. The graphical display should also, of course, represent the current state of the game accurately.
The method you use to play poker is up to you (e.g. rule based, mathematical, heuristics, evolutionary etc.). At the start of the project I will give both teams an identical set of notes that you can use to start your research.
Note - Any changes that were made to this problem description will be declared later in the report.
Brief Introduction
Past research efforts of the Artificial Intelligence community have concentrated on building high performance computer game programs where conditions of certain information, performance and progress can easily be measured. This was first achieved by the ‘Deep Blue Victory’ over the World Chess Champion ‘Garry Kasparov’. The success that was achieved by this program is due to the ‘Brute Force Search algorithm’. As the correlation of search speed to program performance was faster this allowed the program to succeed [ref 1].
However, in recent years, a number of interesting games have attracted the attention of the AI researchers such as ‘Poker’ and ‘Bridge’. These games have been avoided in the past as they have imperfect information (they have hidden information, unknown to opponents). Therefore, the ‘Brute Force Search’ was not successful in this area as it searches game trees hence it works for perfect information games (no hidden factors).
However, Poker has a number of attributes that make it an interesting domain. These attributes include:
• Multiple competing agents (more than two players),
• Imperfect knowledge as the opponent hold hidden cards,
• Risk management that involve different betting strategies and consequences,
• Agent modelling, allowing the player to identify patterns in the opponent strategy and exploring them,
• Deception, which could be bluffing and dealing with unreliable information.
All of these attributes are challenging dimensions to a difficult problem that attracted the computing community [ref 2].
Research started on this area few years ago, but up until now, programmers could not build a program that can play a world championship game. This is due to two reasons, firstly, the limitation of the program and the complexity of the game activities and secondly most of the research has been done on a simple type of game that includes only two players only and one round of betting. However, research continues in the field of creating a high performance Poker playing program.
The remainder of this report will show the work done by our group to complete the task of building an automated Poker Player.
3. Research
We as a group felt that this section was a very important part of the project, as it is vital that we all the facts about poker before we get stuck in to the project.
Various areas of research needed to be carried out, including everything from the history of Poker to the AI modelling of Poker. Also, many members of our group had never played poker before this project was started, meaning we had to learn how to play Poker before we got started on the project to gain a better grasp of the task in hand. Also, Paradise Poker helped in learning how to play Poker [ref 12]. This itself can be classed as research. The majority of the research that was carried out came from Journals that were supplied to our group at the beginning of the project (the same material was provided for the other group) by Graham Kendall, our project Supervisor. Also other means of research were used, which are described further in this section of the report.
The following sections describe the areas of Poker that were looked into in great depth by one or more of us during the early stages of the year.
3.1. Brief History of Poker
There are many different opinions on the origin of poker and where it was invented. The game is best known in America while its origin has not been discovered as of yet. At the same time there is a direct descendant that Poker is likely to be from a ‘Persian’ game called “As Nas”. Although, the first direct reference to the game was found around 1830 in New Orleans, referenced by Jonathan H.Green [ref 3]. However, the game was called ‘cheating game’. Not much later Poker had become a exceedingly popular game throughout the world. At present it has many variations with different rules and strategies but the hierarchy of winning hand remains the same.
3.2. Poker Variations
There are two different types of Poker games ‘Draw Poker’ and ‘Stud Poker’. In ‘Draw Poker’ the player can draw new cards from the deck after the initial cards are dealt faced down. Then the betting occurs after receiving the initial cards and drawing the new ones. In contrast, 'Stud Poker' games are where the player plays with the initial dealt cards as some are faced up and the others are faced down. The bet in this type of game occurs after facing up each new card that is dealt and facing down the last card. [ref 4].
Either type can involve five or seven cards, which can be played in two different ways. First is when poker game is played high where the highest hand wins the pot. The other way is where the game is played high/low as the bet is split between the person with the highest hand and the lowest hand ().
However, there are three main variant types of poker
• Five Card Draw: Where each player is dealt five cards face down. Each player has one chance to throw away any cards in return for a new one [ref 5].
• Seven Card Stud: This game is best for beginners. Each player is dealt two cards face down and one card face up. The player with the lower card opens the betting where each player can then either Call, Raise or Fold. This type of game has another types in it such as Omaha [ref 5].
• Texas Hold ’Em: This game was assigned to the group. As is used to determine the world champion in the annual World Series of Poker played in Las Vegas. It is considered to be the most strategically complex poker variant that is widely played in casinos and card clubs. However, it has been chosen due its simple rules and logistics. However, 'Texas Hold ’Em' is a variant of 'Seven Card Stud Poker' but is was invented to speed up the game and allow more players to participate at the same time[ref 5].
3.3. Rules of the Game
The game is usually played with 8 to 10 players (no less than two players and no more than 10 players). It begins with the pre-flop as each player is dealt two cards, which are faced down, followed by the first round of betting, which is started with two forced bets called the small blind and the big blind[1]. Then, three community cards are dealt face up on the table, which are called the flop, at the same time, the second round of betting occurs. On the turn a fourth community card is dealt face up and another round of betting ensues. Finally, on the river, a fifth community card is dealt face up and the fourth and final round of betting occurs. Here all the players’ turn over their two hidden cards and the five community cards wins the pot. However, if a tie occurs, the pot is split [ref 6].
3.4. Betting Strategy
'Texas Hold ’Em' uses a structured betting strategy as the betting increases the pot and indirectly gives information about players and their hands. However, the amount of each bit is strictly controlled in each betting round. There are two denominations of bets, called a small bet and a big bet, which will be $1 and $3 in this project. In the first two betting rounds, all the raises are $1, while in the last two rounds, they are always $3. In general each player has one of three betting actions they can either:
• Fold: As the player is no longer (in his/her opinion) capable to win the pot. The player is out of the game and looses all the money that has been contributed.
• Call/Check: A player calls to match the current pre-player contribution in order to be able to continue the game. However, if the amount of call is zero then the player checks.
• Raise/Bet: The player increases the current pre-player contribution. In other words the betting in the round starts.
In general, there is a maximum of three raises per betting round [ref 7]. The betting option rotates clockwise until each player has either matched the current bet or folded.
3.5. AI Computer Modelling Methods
By the researches that were done, we found that there are a minimum set of requirements for a strong poker playing program. These requirements are not independent thus they are continually refined as new capabilities added to the program. This includes hand strength, hand potential, betting strategy, bluffing, unpredictability and opponent modelling. A brief description of each characteristic will be given. This is taken from [ref 8].
• Hand strength: Assesses how good a hand is compared to other possible hands. It is calculated by considering the number of players, their position at the table and the history of betting for the hand. At the time it calculates the probability of the opponent hand to give more accurate calculations.
• Hand potential: The probability of a hand improving as more board cards dealt. For example, having four cards of the same suit would give you a low hand strength but a high hand potential as and further card from that suit would give you a flush. Therefore, a hand potential is a function of your cards and the current community cards.
• Betting strategy: Deciding whether to Fold, Call or Raise at any given point, based on a minimum hand strength, but can be improved to consider other information such as pot odds, hand potential, bluffing, opponent modelling and unpredictable play.
• Bluffing: Allowing the player to make a profit from weak hands. This can be used to create a false impression about your play and to improve the profitability of subsequent hands. Therefore, it is essential for successful play.
• Opponent modelling: Allowing the player to determine a likely probability distribution for the opponent’s hidden cards or betting strategy. This uses a single distribution for all opponents in a given hand. However, it can be improved by modifying those probabilities based on collected statistics and the betting history of each opponent.
• Unpredictability: makes it difficult for opponents to form an accurate model of the player strategy by varying it over time.
However, there are several other characteristic which may not be necessary to play strong poker but very relevant for a World Champion. [ref 1].
In our program we plan to implement as many of these as time allows. The use of the ‘Poki libraries’ from the ‘University of Alberta’ means that some work has been done for us in analysing the cards. As ‘Poki’ can measure the aggressiveness of each player, and use this information to make better inferences about the implementation of each observed action. The main classes that will be used to determine what play to make are HandEvaluator, HandPotential, ProbTriple, and ActionFrequency. Together, they can calculate your hand strength and hand potential, give your player degree of unpredictability and aid in modelling opponents.
The ‘University of Alberta Paper’ also describes a method of calculating betting strategy and pot odds. As it says that in pot odds, you use your chance of winning the hand, and the amount of money in your hand to decide whether to fold, call or bet. Here’s what they say about it:
“Pot odds are the comparison of your winning chances to the expected return from the pot. For example, if there is only a 20% chance that we have the best hand on the river, should we fold, call or bet? The correct answer is that we have not given enough information to answer the question. Assume the pot contains $100 after the only opponent bets $20. If you call in this lose 4 times out of 5, for additional cost of $80. However, you will win 1 time out of 5 for a profit of $100. Therefore, under these assumptions, you should call, resulting in an average profit of $4 per hand. However, if the pot only contained $60, you should fold, since calling would yield an average loss of $4 per hand.” [ref 9]
3.6. GUI - Design options
The appearance of the Graphical User Interface is not the essential part of the project although it does have its importance. The Front-end must have the basic features that a normal GUI for a Poker table would have. An image of a table with images of something representing players i.e. chairs, cards, poker chips, etc. The features of the normal Poker game should also include a Dealer, Fold, Raise, Check buttons and follow the rules of the game to play a game of Poker, which in our case is the variation of 'Texas Hold ’Em'. Therefore, the group has decided to create a simple GUI as it will be faster and in turn be user friendly.
The main aim of the GUI is to imitate the game that is being played and tell the computer player the options of what to do next. This will allow the player to keep a track about the status of the game and hopefully play a good, tight game of Poker.
4. The Back-End
4.1. The Need for a Back-End
Splitting the project into two separate modules provides a neat way of distributing the workload. The idea is that, given a standard communication protocol, the Back-end and Front-end can be developed independently with the safe knowledge that the two will communicate when 'plugged' together. The Back-end traditionally refers to the unseen work that a program does, while the user interface or data collection module is known as the Front-end. There are many parallels between this architecture and the now famous Client-Server architecture synonymous with the World Wide Web. Indeed a properly designed program split into a Front-end and Back-end should lend itself to this type of application with very little effort.
4.2. Research
We have been given full use of the ‘Poki libraries’, a mass of extremely useful open source code taken from the famous ‘Poki’ online poker player developed by Jonathan Schaeffer from the ‘University of Alberta’. All of this source code is provided in the Java programming language, the language we have already decided to use, and can be found here . The libraries were at first extremely overwhelming and it appeared that they were far too complex and elaborate for our use. However after many exhausting hours of unravelling the code and gaining a vague understanding of their operation it became clear that as they were they provided a fully operational text based game of Poker. All that was required was a simple instantiation of the main Dealer class and for some players, either random or human, to be added to the game, see Appendix Bi.
This was a breakthrough as we now had an established way of representing cards, managing betting and keeping logs without having to reinvent the wheel. Hence, more effort can be put into understanding and refining the way the ‘Poki’ code makes it decisions, with the intention of making a better poker player.
4.3. An Example Back-End
At this stage the poker player operated randomly. On creation of a new RandomPlayer, we pass 3 floats to the constructor. The first describes the probability the player will fold at a given stage, the second that it will Check or Call, and the third it will Raise. Therefore passing suitable variables at runtime can make an aggressive, weak or balanced player. It was made clear at the beginning of the project that the most fundamental decision we would have to make would be the way we separate the front end from the back end. This essentially meant we would have to decide what would contain the main bulk of our poker program the GUI (the Front-end) or the Back-end. Although the decision was arbitrary, (any combination could produce a successful result) we decided to make the GUI lightweight and put the engine into the backend. This decision was made for the following reasons:
1) Java GUI’s are slow enough as they are.
2) Any slight changes to the Front-end are unlikely to upset the critical poker engine.
3) We can test a variety of GUI ideas using the same back engine to come to a
final decision.
By splitting the program into two parts like this we can also split the programming tasks. For this to be done we need to draw up an explicit specification describing the "hooks" the back end must provide for the GUI to attach successfully.
4.3.1. A Working Model
We wanted to see just what was possible from this ready-made poker player and so we made an example Back-end program by modifying some of the classes to give us the functionality we required.
We need to let the Back-end know the current state of play and keep it updated with the Paradise Poker game we have joined. This includes the number of players, their names and positions, the current dealer (or the button position) and the amount of cash owned by all players. Entering all this information line by line at appropriate prompts proves quite fiddly in a text-based representation, but at least we can get an idea of how the final Back-end will work without the additional complications of the GUI. All this initial information can be implemented in the main method in our code (see Appendix Bii). At the moment we are assuming that no players can leave the table or swap positions.
To provide a way of giving any player the button, we override the playGame()method in the Dealer class to accept an integer as a parameter. This extra variable is used to synchronise the current player in the local game with that of the game at Paradise Poker.
We also have to provide a way of telling our engine what cards are in play at the Paradise Poker table. We need to tell it what cards are on the board and what cards we have been given. This provides the most radical shake up for the classes as we must take all the randomised dealing methods and replace them with methods that will receive an input of cards. The ‘Poki’ class provides a very efficient way of representing all 52 cards by two characters and their aliases are listed below:
Suite:
Clubs - C
Diamonds - D
Hearts - H
Spades - S
Rank:
1-9 - 1-9
T - 10
J - Jack
Q - Queen
K - King
A - Ace
E.g. The Ace of spades would be represented as As,
The Ten of hearts would be represented as Th.
Each card also has an integer value depending on the card's worth in a game of Poker this is later used by the HandEvaluator classes to help the engine make its decision. To provide a ‘quick and dirty’ way of letting the engine know what cards Paradise Poker has dealt we added a method called getPPCard()to the Deck class (Appendix Biii). This method performs 2 operations:
1) Get a 2-character card representation from the keyboard
2) Remove this card from the engine's own deck
This method then replaced the deal()method in our overridden playGame(int Pos)method. Now instead of randomly dealing a card automatically from the game deck, the engine stops and asks for input. This input will be the hole cards we have been dealt and the cards that have appeared on the board. By removing these cards from our own private deck we can later make a more accurate guess as to what the other players have.
4.3.2. Perfect Information
Appendix Biv shows what this simple Back-end can achieve. The output shows a game between Jesse, our program, and a human player James. This player could be someone sat at a Paradise Poker table. Since the action moves so quickly at these tables, we decided to input the other players cards ourselves, so at the moment we have perfect information on the state of play. At Paradise Poker we do not have this luxury and so this is not an entirely accurate simulation, however as stated before this is only a temporary experimental Back-end, purely for development as we get the basics working. The need for a slick and efficient GUI is evident as the task of inputting every card dealt and every player's move in this way is very cumbersome.
At the moment our engine is evaluating hands and deciding on victors itself, this is not going to be the case in future versions. The engine will be tailored so that its conclusions are kept to itself and the only outputs it has are its own suggestions, Fold, Check or Raise, at the appropriate stages.
4.3.3. Obvious Limitations
Other improvements to be made are:
1) The method getPPCard()must be rewritten to accommodate input from a GUI,
2) There must be a cleaner way of setting the button (from a GUI),
3) There is currently no Undo feature:
This is a critical feature of the Back-end. As the pace of play at Paradise Poker is furious data entry mistakes may be made in the heat of the moment - possibly confusing the engine and ruining chances of success.
4) No real work has been done on improving the engine's basic game of Poker:
Current ideas for this engine include improving the player logging to provide a history for our player to refer to. Also the engine should be able to dynamically alter its balance of play. For example, if we are loosing consistently to weak hands the probability that we Call or Raise should be increased
4.4. Developing the Model
As the University of Alberta ‘Poki’ classes provided such a comprehensive implementation of a round of Poker, it seemed foolish not to take advantage of its reusable Object Oriented structure for our own ends. Therefore using the above working model as a base the task began to strip out the superfluous functionality it provided and putting in what we needed.
4.4.1. Stripping the Back-End
The ‘Poki’ classes provide much functionality that our Back-end simply must not have. First of all the ‘Poki’ classes were designed to not only play legal poker but also to manage a game. This means that the class deals the cards to every player, it decides where to place the button and it performs "hand evaluation", or deciding who wins. This is all superfluous to our needs as this is all done by the game manager at Paradise Poker. A way was need to strip out this functionality, and in effect leave the classes playing Poker, but with only seven cards.
Rather than break the classes up completely and lose many useful functions, it was decided to have the ‘Poki’ class "mirror" the game-taking place at Paradise Poker. The Back-end would effectively be autonomous once started, simply accepting cards and initialisation data from Paradise Poker at certain points in the game, chiefly:
1) Number and position of opponents,
2) Position of the Button,
3) Dealt cards (Hole and Board cards),
4) While playing the game itself, keeping in step with Paradise Poker.
The major difference is that the classes would not know what cards the other players had, leaving the AI making an educated guess based on the cards currently visible. To implement this feature, the Dealer class was modified to firstly deal "null" cards to the opposition and then ask for two cards from the user, representing the cards dealt by Paradise Poker. Then a round of betting, taking action once again from Paradise Poker, followed by the flop, turn and river cards all provided by the user. Also the showdown functionality needed to be removed to avoid null pointer exceptions when comparing cards that were in effect non-existent.
The result of these modifications was a skeleton Dealer class which provided all our needs concerning the requirement of legal Poker.
4.4.2. Modifications to Accommodate a GUI
There are two occasions in the modified Dealer class when input is required and the back-end must take it upon itself to prompt the user. We have already seen in section 4.1.1 how getPPCard() received input via the keyboard in order to return a card to the Dealer. Although suitable as a method for testing, the input is relatively slow compared to the speed of play at a Paradise Poker play money table. A faster input method would undoubtedly make use of a GUI, and more information on the development of this interface can be found in Section 8 - Developing the GUI.
4.4.2.1. Getting a Card
The method getPPCard() needed to be altered so that instead of waiting for keyboard input, it waits for a mouse click on the 'Card Window'. The ‘Card Window’ itself it created as soon as the PPDeck is instantiated. This enables us to have a new set of cards for each deck that we may create. When we are ready to receive a card we pass the GUI a “GetCard” message and wait for a card from CardWindow. This event is signalled by a “CardSelected” message deposited in the MessageArea. For more details of this mechanism see Section 9 - Back-End & Front-End Communication. Once this message is received the String representation of the card is also collected from the MessageArea and execution can continue. The code is shown below:
public class PPDeck extends Deck {
// constructor and initialisation
...
public String getPPCard()
String action = PokerGUI.theExchange.getAction(true);
// tell the GUI we are expecting a card now
PokerGUI.theGUIExchange.put("GetCard");
while(!action.equals("CardSelected")) {
action = PokerGUI.theExchange.getAction(true);
} // wait for the message
theCard = PokerGUI.theExchange.getMessageData();
// create card from String and extract from deck
...
}
} // end class
The method getPPCard()also performs another important operation. It extracts the card from the deck of cards it keeps in memory. This is vital for AI routines, as they will know what cards other players cannot have if they are visible or we have them.
4.4.2.2. Getting a Decision
Once again the working model (section 4.1.1) used the keyboard to get a character representing each player's decision. This method was easily altered, to replace the keyboard prompt with a graphical dialog box. Although this is not the most elegant solution it is under the circumstances the most convenient and possibly the most effective. The Player class was extended to provide a ParadisePlayer class. This class was identical to a HumanPlayer class, except that the prompt was graphical instead of console based. The implementation is as follows:
public class ParadisePlayer extends Player {
// constructor and initialisation identical to HumanPlayer
...
public int action() {
while (true) {
Object[] options = "Check",
"Raise",
"Fold";
int n = JOptionPane.showOptionDialog( null,
gi.getCurrentPlayerName() + ": "
+ "What is their move? ",
"Move required!",
JOptionPane.YES_NO_CANCEL_OPTION,
JOptionPane.QUESTION_MESSAGE,
null,
options,
options[0]);
switch(n) {
case 0: return Holdem.CHECK;
case 1: return Holdem.RAISE;
case 2: return Holdem.FOLD;
}
}// end while
}// end method
}// end class
The adaptation of a "Yes, No, Cancel" dialog box to a "Check, Raise, Fold" box is effective. The move is returned via a switch statement on the user's response and having received a decision from a player the Dealer can continue.
4.4.3. Putting it all together - the Back-End Class
The Back-end class encapsulates all the elements of a ‘Poki’ game and tailors them to our needs. It deals with game initialisation, collecting data from the GUI such as player names and button position and pipes them through to the Dealer class that runs a customised game of ‘Poki’. This class is run as a thread along with the GUI, which is both, called from the executable Poker class. When the game of Poker has finished, the Back-end thread dies and depending on its closing options the Poker class respawns the Back-end or the software is terminated and the GUI also dies. If the Backend is respawned the GUI is reset, any previous game data is restored and the user is allowed to modify it before resuming play.
public class Backend extends Thread {
public Backend() {
// constructor initialises and resests the GUI
}
public void run() {
Dealer theDealer = new Dealer();
// collect player names from the GUI
// and add players to the Dealer class
...
// collect the button from the GUI
...
// start the game
theDealer.playGame(buttonPosition);
// reset
reset=true;
// tell the GUI we have finished
...
// die
}
}
4.4.4. Putting it all Together - The Poker Class
The Poker class has the responsibility of creating multiple instances of the MessageArea, the Backend and of the GUI. Once this is done, their threads are started and the class monitors messages from the Backend and passes them to the GUI. Messages from the GUI to the back-end are picked up either by the Backend itself or the class waiting for the message. This is all done in the runPoker()method.
If a “ResetTable” message is received by the Poker message handler the runPoker() method returns to the main()method, which calls the runPoker() method, thus restarting the game.
When the “ExitGame” signal is received the while loop in the main()method terminates and the Poker class exits to the system. This operation is shown at its simplest below:
public class Poker {
boolean exitGame = false;
public static void main(String args[]) {
//create instances of MessageArea and the GUI
//start the GUI
...
while(!exitGame) {
runPoker();
}
System.out.exit(0);
}
void runPoker() {
// create an instance of the Backend
// start the backend
...
// monitor messages
// if Exit message is received exitGame=true
...
}
}
5. Building a Computer Poker Player
The game engine is at the very core of a poker-playing program. It is what makes the decisions about what to do at any point in the game. By using the ‘Poki’ library from the University of Alberta, the game engine can be separated from much of the rest of the code and programmed separately. The class that allows this is the abstract class, Player. This represents one player in a game of poker and defines methods to allow interaction with the game:
newGame() is called when a new game is started, passing an instance of GameInfo, the two player’s two hole cards and an ID number for the game.
update() is called by the Dealer every time something happens in the game. This might be another card is dealt to the board, or a betting decision from another player.
action() is called when a decision is required from this player. It returns an integer representing its decision to fold, check/call or bet/raise.
The University of Alberta includes two concrete definitions of Player in the libraries, and these are useful to demonstrate how the system works. RandomPlayer is a player which decides at random whether to fold, check/call or bet/raise.
The instance of GameInfo, passed as an argument to newGame can be used to find out more about the current state of play. GameInfo has methods which return the number of players left in the game, the current amount in the pot, what cards have been dealt to the board and other information about the game which is vital when producing a poker engine.
The RandomPlayer also has an instance of ProbTriple, a class that defines a probability triple. This is a set of three floating-point numbers, the sum of which must equal 1. This can be accessed by passing the select() method a random number which will return a decision of whether to fold, check/call or bet/raise based on the probability triple’s odds. In this class it is used as the sole basis of decisions, but it can also be used to add a degree of uncertainty to a poker engine’s play.
5.1 Poker Theory
There are many techniques that can be implemented in a poker engine to let it play a good game. Some of these are fundamental to playing a sensible game, whilst others are non-essential, but increase the player’s ability, especially against human players.
Hand strength is probably the single most important calculation to make about a game. It is a measure of how good your hand is compared to all other possible hands. So at the low end of the scale is 9 High, and the highest possible hand is a Royal Straight Flush. The Poki library has a HandEvaluator class which contains methods to find the best 5 card hand from a given set of cards, rank your hand compared to all others, name a hand, and calculate the chance of winning the hand.
It is this last method, handRank(Card, Card, Hand), which we use in our game engine to determine hand strength. It returns a double representing the probability that a given hand will win against one other player. If there is more than one other player left, raise the probability to the number of other players to get the probability that your hand will win against everyone.
Hand potential is a measure of how much your hand can improve with the addition of another card. For instance, the hand 10 J Q K of clubs has a low hand strength – King high – but it has a high potential for improvement – any club would form a flush and a nine or an ace would form a straight. Hand potential is used before the river, when there are still cards to be dealt. Hand potential is not usually used to decide whether to bet/raise as you probably don’t have a winning hand, but it can be useful when deciding whether to fold a hand or call to see the next card.
The Poki library provides a HandPotential class, which can determine the potential for improvement. It can also be used to calculate the negative potential. This is the probability that, given you are currently ahead, future cards will put us behind. How hand potential is calculated is described in (Denis Papp, Dealing with Imperfect Information, M.Sc. thesis, 1998).
Pot odds is used to determine whether it is worth calling to see an opponent’s hand or if you should just fold. It is generally used when the chances of having a winning hand are low and have decided in principle to fold [ref 9] gives an example of how pot odds works:
“Assume the pot contains $100 after the only opponent bets $20. If you call in this situation, you will lose 4 times out of 5, for an additional cost of $80. However, you will win 1 time out of 5 for a profit of $100. Therefore, under these assumptions, you should call, resulting in an average profit of $4 per hand. However, if the pot only contained $60, you should fold, since calling would yield an average loss of $4 per hand.”
Uncertainty is not our program’s inability to decide what to do, but is a way to prevent opponents from easily developing a model of our play by using a probability triple. Instead of making a concrete decision about what play to make, we decide on which probability triple to play with and use that to make the final decision. For instance, instead of straight raising when we have the pocket rockets, we might decide to use the probability triple fold=0.0, check=0.2, raise=0.8. This makes it much more difficult for opponents to build up a picture of what hand we have – they cannot determine from the fact that we raise we have a good hand. Conversely, we can use a probability triple with a chance of raising with a mediocre hand.
Bluffing is slightly different to uncertainty. Whereas uncertainty is where we randomly go against our initial decision, bluffing is where we use our knowledge of the cards dealt to fool other players into thinking our hand is better than it actually is. For instance, if three clubs are dealt on the flop we could raise aggressively in the hope of fooling opponents into thinking we have two more clubs in the hole. Bluffing is difficult to implement well as we must also take into account what our opponents have and not get drawn into our own bluffs.
5.2 Implementation
Our implementation of the Player class is called JessePlayer. It is based on RandomPlayer but has a rewritten action() method. JessePlayer implements several of the tactics described above. Different combinations of tactics are used at different stages of the game. For instance in the last betting round, after the river card is dealt, hand potential is not used to determine betting strategy as our hand cannot improve any more.
The most important tactic, and the one that is used the most, is hand strength. The decision our program makes depends upon the chance of winning the hand. If there is a very strong chance of winning, JessePlayer will decide to bet/raise. As the chance of winning decreases, we will check/call and then finally fold. Probability triples are used wherever we make a decision. There are a predefined set of triples ranging from STRONG_RAISE to STRONG_FOLD which make our play harder to predict. If a probability triple returns a decision to fold, we first check to see if any money has been bet. If not, we can check for free.
If our hand is not good enough for JessePlayer to decide to bet/raise or check/call straight away, it will go onto the next level of decision-making. In betting rounds before the river, where there are still cards to be dealt, it uses hand potential to determine whether it is worth checking/calling or if we should cut our losses and fold that round. This works in a similar way to hand strength; the probability of our hand improving determines what probability triple we use.
6. Front-End Design - GUI
6.1. Initial Ideas
As mentioned in Section 3.6.GUI - Design Options, we as a group felt that the GUI (Front-end) was not an essential part of the project overall. However, it is important to the extent that it must be clear and user friendly, as information has to be extracted from the GUI quickly and efficiently. Overall the main aim of the GUI is to imitate the game that is being played and to tell the computer player what options there are for its next move. This will allow the player to keep a track about the status of the game.
A quick brainstorm was made of the main features that would need to be included on the GUI Appendix A.
The main features would include a main table with chairs imitating the player with some kind of plaques with the players name on and chip value (in a numerical value). Also, there needs to be something that would represent the dealer and most importantly our computer players cards and the flop cards needs to be shown.
As the GUI was required to be simple we thought that we could represent the cards using text, however it was decided against as it would make it more difficult for the users to recognise the cards in comparison to simply seeing the cards.
After we had discussed the main aspects that were required on the GUI, Viral and Wafa went away and drew a rough outline for our design for the GUI on paper, Appendix B.
6.2. Mock Up Design
The next stage to the design of the GUI was to create a mock up design of design using the software application Visual Basic. It was decided to use this particular package as Viral had used it before and we just wanted to see what it would look like on a computer screen. This design in V.B. was not going to be used for the final GUI as it had been decided that the GUI would be created in Java.
[pic]
0
7. Creating the GUI
7.1. Initial Design
After completing the design of the GUI, Section 6. Front-End Design – GUI, we started to write a Java based GUI (written in Java Swing [ref 10]) similar to the one we produced in Visual Basic, however, there were subtle changes that had to be made to improve the GUI.
As the project’s requirement, this GUI is developed for an automated poker player that plays ‘Texas Hold 'Em.’ The GUI should therefore satisfy the following requirements:
1) The GUI must be as lightweight as possible since a Java GUI is slow,
2) Have a section to display the current state of playing cards on the table. Both texts and images
will do, but images are preferred as it is much more user friendly,
3) As the program is designed to accept the maximum 10 players, these players should be distinguishable.
4) It’s clear that each player should take turns to perform the functions of either ‘Fold’, ‘Check’ or ‘Raise’,
5) A summary of every play step should be recorded and displayed, which is refreshed after every
performance,
6) A user of the program should be able to clear the table, reset the game and even undo a step.
Using the fillOval method, a green oval was painted to the background of the window frame, which would act as the main play area of the table. On top of the table, a JLabel was used to hold a dealer and some chips images. To represent a player on the table, we firstly used a JLabel as well to display the users information such as name and chip value. Then, right below that label, there were three buttons: Fold, Check and Raise that would perform their corresponding functions when connected to the Back-end.
Then such player panels were put along the edge of the table. Below the oval, we created another panel to hold a JTextArea and three other buttons. The text area was used to display any information on how the program and the game were progressing including comments from the dealer and the players. The three buttons were “Undo”, “Reset” and “Clear Table”. Finally, we managed to download some poker card images from the web, and used a JLabel again to display them in the centre of the table.
The first attempt of creating the GUI is as follows:
[pic]
However, we all felt that it could be improved vastly. This GUI was analysed further and comments were made on what could be done to improve the look of the GUI. The following are the improvements that could be made:
1) The appearance of the GUI is very basic. The playing table, player seats, buttons and the background should be improved,
2) Currently components do not work, code should be added to them to provide interaction between the front-end and the back-end.
3) A method of input, such as a text box should be added.
Summing up all the points, a second attempt is developed as below.
[pic]
This is a significant improvement upon the first attempt. The table and chairs have been given a 3D look, which were produced using the 3D Studio MAX software. The buttons have been altered, reducing groups of “check”, “fold” and “raise” buttons to a single panel, which is positioned below the table. Some buttons like “reset”, “clear table” and “fold” can interact with the GUI. An input box has also been added, which can accept a string input to display the poker cards and enter the player information. Also, the hole cards for the automated player can be shown in front of the table.
This front-end program is developed separately from the backend. Many other aspects have to be considered during programming. We need to consider a suitable object-oriented style and make it easily portable to the back-end. This may require a specification of the critical points needed to make successful connection with the back-end. Considering that the GUI design is fully dependent on the back-end program, some significant changes to back-end may require modifications to the GUI, hence the final version of the GUI should be produced after the final version of the back-end.
8. Developing the GUI
8.1. Limitations of the Interim GUI
The GUI presented at the interim stage of the project was simply meant to give an impression of what a final interface may look like, and what features it would provide. To this end it was successful, it did not however provide anything useful in terms of functionality. A few of these shortcomings are listed below:
• The GUI provided no ‘hooks’ to the backend.
• Objects on the table were ‘static’ in the sense that they could not be modified at run time. This included cards both on hole and board, players and the scroll pane.
• Buttons were inactive.
8.2. Developing a Front-End
It was important to address the issues outlined above, and provide methods of interaction convenient not only to the user but also to the Backend. Having spent much time becoming familiar with the interface used at Paradise Poker it was obvious that any GUI would have to enable the user to input data quickly. This is because the speed of play at the tables is generally very fast.
The GUI needed to provide functions in order to provide the Back-end with data and represent the Back-end to the user. The most important functions and the approaches to them are explained below.
8.2.1. GUI Layout and features
[pic]
The GUI is divided into two areas the MainArea and the CardWindow.
MainArea: is made up of 10 seats, a scroll pane, 3 buttons and 7 card icons.
CardWindow: contains 4 separate panes, accessible by a tab widget. These pains each contain a suite of cards.
8.2.2. Interaction with the User
To make the software more user friendly and to ensure the user knew what data the program was expecting it was necessary to include some method of Back-end feedback in the GUI. Once again the client distributed by Paradise Poker gave the idea of using a scroll pane to record significant events that have happened in the game. We also decided to use this scroll pane to ask the user to interact with the GUI. For example the scroll pane will display the following when expecting to receive information about new players:
Jesse: Please enter player names.
We decided to use the “Jesse:” form so it was clear that Jesse, the name for the backend code was expecting the data and not some external event. The method addConsoleText(String message) appends text to the pane, whereas the method setConsoleText(String message) replaces all text in the pane with the supplied string argument. The pane adds recent messages to the bottom and earlier events can be read by using the scrollbar to scroll upwards.
8.2.3. Establishing Initial Game Data
To begin a game of poker, the modified Back-end needs to know three pieces of information.
1) Who is playing the game
2) The position of the players relative to each other
3) The position of the button
Users were previously added to the current game purely using the console. The Back-end would ask how many users were playing the game and then ask for their name. Although the solution was effective it would clearly not be practical in a graphical environment. We needed to provide a way of informing the backend of users and their positions using a graphical interface.
Since our Interim Front-end featured a remarkable similarity to the layout of a Paradise Poker table, it was decided to provide a similar interface for adding players. The user should be able to select a seat on the table with the mouse and enter the player’s name. On confirmation of the name the details should then be passed to the backend for insertion into the game. The GUI should also pass on the player’s seat number so that player can later be referenced by the Back-end. This number can be worked out by the GUI, the seat positions are shown below:
[pic]
It is important that the Back-end has a seat at the table. This position is flagged by entering the reserved player name “JESSE” at the required seat. When the Back-end sees this string it places itself at the table, makes a record of its own seat number and waits for its turn in the game.
With player numbers passed from the backend as above, the GUI can now reference the current player to their position in the game window. This is crucial to making player names highlighted on their turn, or not highlighting a name if that player has folded. The GUI signals active players by highlighting their name in bold type, inactive players in normal type and folded player names are greyed out.
The example below shows that the player “tim” is required to make a move, the player “JESSE” is still in the game and the user “mike” has folded.
[pic]
When all the players have been added the user must inform the Back-end by pressing the “Confirm” button. The GUI will then ask for the ‘dealer button’ to be placed. This is done by clicking the seat, which corresponds to the button player on Paradise Poker. The button can be repositioned until the “Confirm” button is placed. Then the game begins.
8.2.4. Representing Dealt Cards
With a game underway the first job of the GUI is to tell the Backend what cards we have received from Paradise Poker. This information is input via the card window. The card window represents a full deck of cards and is shown below:
[pic]
Since cards are dealt very quickly at Paradise Poker, an efficient method of accessing 52 cards was needed. Many interface implementations were tried but the most successful seemed to be through the use of tabbed panes. Each pane represents a suite containing 13 cards. Switching between suits is a simple matter of clicking the desired tab. The cards are represented by open source .gif images. Clicking on a card causes messages to be sent to the Back-end, which should already be waiting patiently for a “CardSelected” message. Once the card has been selected it disappears from the suite ensuring that it cannot be reselected. This state is shown below:
[pic]
Since the card window represents a current Deck class, it seemed fitting to extend that class, creating a new subclass called PPDeck. When instantiated a new card window is created and displayed along with a new instance of the Deck class. The implementation is as follows:
public class PPDeck extends Deck {
public CardWindow myCardSelector;
public PPDeck() {
super();
// create and display a new Card Window
myCardSelector = new CardWindow();
}// end constructor
public Card getPPCard() {
// handle GetCard message from myCardSelector
…
}// end method
}// end class
8.2.5. Establishing User Decisions
User decisions are required when we need to know what move a player has made at Paradise Poker. This could be one of three options at any stage of the game, Check, Raise or Fold. As the backend is autonomous, in that in effect it is playing an independent game of poker, it seemed easier for it to retrieve a user decision itself. The limited choice of options and the necessity of a response indicated that the data could best be obtained through the use of a dialog box that popped up with the three available options. This dialog box is the only time where the Back-end creates and displays a GUI element on its own. The decision was made for both convenience and more importantly immediacy.
The class that creates and displays this dialog box is an altered version of the HumanPlayer class. A graphical dialog is displayed instead of a console prompt. The dialog box appears as follows when a decision is needed for a player.
[pic]
A constant is returned by the dialog box and another appropriate constant is returned to the Back-end represent the user’s selection.
8.2.6. Board and Hole cards
Board and hole card are represented by icons containing relevant cards. The cards initially appear on the screen face down. The cards are stored in an array and their index represents the order they are selected.
Card Index Card Represented
0-1 Hole cards
2-4 Flop cards
5 Turn card
6 River card
A counter is kept in the PokerGUI class, which stores how many cards are visible on the screen. If the counter is zero then the first card to appear will be the 1st hole card. If the counter is 4 then the next card to appear will be the Turn card. When the card needs to be drawn, the image of the appropriate card is loaded into the array at the specified index and then rendered to the screen. This is all performed by the addCard(String theCard) method in the PokerGUI class.
8.3. Implementation
8.3.1. Implementation Details
The GUI is created and initialised by the PokerGUI class. This class creates everything see in the MainArea. The class makes extensive use of the Java Swing package, distributed with Java. Below follows a list of graphical elements created by this class and their role in the GUI.
Class Var Name Description
JPanel mainArea Contains all elements in the MainArea
JButton confirmButton Creates a button to confirm actions
resetButton Creates a button to reset action
exitButton Creates a button to exit the software.
JTextArea consoleText Holds status text.
JScrollPane consoleScrollPane Scrolls data in consoleText.
JPanel south Contains elements at lower edge.
JPanel[] PlayerPane[] An array of panels representing seats.
JPanel theBoardPanel A panel holding 5 board cards.
JPanel ourCardPanel A panel holding 2 cards.
Panels theBoardPanel, ourCardPanel and PlayerPane call on classes Board, OwnCards and PlayerPane to create any sub elements contained in the panels. This will include things like IconImages to represent the cards or JLabels to show a player’s name.
8.3.2. Back-end & Front-end Communication
The only purpose of the GUI in this software is to collect information and display information. The Back-end will act on this information. However a mechanism is necessary to pass the collected information to the Back-end and any received information should be displayed. The messaging system is a key part of this communication and is outlined in more exhaustive detail in Section 9 - Back-End & Front-End Communication.
Messages destined for the GUI are handled in the Poker class, which alters anything necessary in the PokerGUI class.
9. Back-End & Front-End Communication
9.1. Background to Message Passing
With the Front-end and Back-end being developed at different paces it became increasingly clear that the two modules of code would need to find a mutually convenient way of keeping each other in step with the current state of play. The Back-end would have to respond to any user action on the GUI and likewise, the Front-end would have to respond to changes sent from the Back-end. Such events are summarised in the table below.
Event Front-End Action Back-End Action Message Direction
Adding a player Draw new player Respond to front-end Front-end -> Backend
Placing the button Handle mouse click Respond to front-end Front-end -> Backend
Reset game Handle mouse click Respond to front-end Front-end -> Backend
Jesse folds Responds to backend Fold player Backend -> Front-end
Selecting a card Add card to table Respond to front-end Front-end -> Backend
Button moves Respond to backend Set new button player Backend -> Front-end
Player change Respond to backend Change active player Backend -> Front-end
Check /R/Fold Handle mouse click Respond to front-end Front-end -> Backend
Table 9.1.1: Examples of 'Message Passing'
Almost every action that could be performed will have some accompanying data. For example, when placing the button we need to know where the button has been placed. Again, if a card has been selected we need to know which card we have been given.
9.2. Research
Message passing is a common term for passing tokens, or items of data between two classes or objects to make an event known. Message passing is common in GUI's. Microsoft Windows, for example uses what is known as a "Message Pump". Anyone who has ever programmed using the Microsoft Foundation Classes will be familiar with this mechanism. The following text is taken from and provides an excellent description of message passing.
“An event-driven operating system, Windows sits on the computer monitoring the hardware for events. Whenever something interesting happens like the mouse moving, Windows detects it and passes the event information on to the applications it's hosting. The event information is called a message. In this case it's the WM_MOUSEMOVE message. If the message is important to a specific window on the screen, that window handles it. For instance, if Windows detects the selection of a menu option in a window, it passes the information to the window's message handler. The message handler contains a case that handles the menu option”.[ref 11]
But simply, Windows spends a vast majority of its time sitting in a "tight loop" waiting for external events (such as mouse clicks, key presses etc) to happen. When they do occur messages are passed to appropriate event handlers and any data let by the event is retrieved.
Since the Back-end will be waiting for the Front-end, or vice-versa depending on the game's stage, we need to implement a similar system. A loop continuously checking for messages, and then handling them as necessary.
9.3. Control
9.3.1. Back-End to Front-End Message Passing
It needed to be decided which module would control the game play. Either the Front-end would be 'lightweight' and respond to backend messages, or the Back-end would be 'lightweight' and respond to the Front-end. The decision proved to be difficult as Table 9.1.1 implies. There are solid arguments for and against both approaches. The University of Alberta ‘Poki’ classes provide a solid Back-end, which could be easily adapted to handle messages from the Front-end. Returning the messages however, would prove tricky for reasons discussed later. The problems would be reversed if the GUI were to handle messages from the Back-end. It was obvious that there would be no elegant solution with two separate classes updating each other's data members as they saw fit. The example below highlights some of these caveats. This code is an outline of the first attempt at interclass communication.
public class PokerGUI extends JFrame implements ActionListener {
public static void main(String args[]) {
Backend myBackend = new Backend();
// code to draw GUI elements
myBackend.start()'
while(!exitButtonClick) {
if (checkButtonClick) {
myBackend.myDealer.checkCurrentPlayer();
}
else if(raiseButtonClick) {
...
}
// other events handles here
}// action listeners
}
}//end class
Source code 9.3: First attempt at interclass communication
At first glance the solution looks effective, however there are some glaring pitfalls. First of all if this implementation had actually compiled, it would have been theoretically possible to check a player at anytime, even if play had stopped or if we were unable to check.
Secondly the Back-end has been constructed in a professional object oriented manner. The line of code:
myBackend.myDealer.checkCurrentPlayer();
is completely invalid. The University of Alberta ‘Poki’ source code defines the Player class as Protected, which means that decisions relating to that player cannot be made outside of that class. This is exactly what is attempted in the above section of code. It was suggested and pondered at many informal meetings if we could modify and in effect rewrite the University of Alberta classes to our own ends, making necessary fields and methods public or static where appropriate. This it was said would allow much broader access to these fields and make implementations such as the above possible. However this idea was eventually discounted for two reasons:
1) The OO Design of the classes is well implemented, and implemented for good reasons. Violating this design for a GUI 'hack' seems very cavalier, especially when GUIs lend themselves to object orientation so conveniently,
2) Overlapping front-end and back-end in this fashion seemed inappropriate given the group structure.
Also, with this implementation, there is no mechanism for the Front-end to respond to the Back-end. The class PokerGUI creates an instance of Back-end, called the Back-end. In object orientation there is no way of referring to PokerGUI from within the Back-end, unless all of PokerGUI becomes static, which is undesirable. This problem will arise everytime a message passes from the backend to the Front-end (Back-end->Front-end, in table 9.1.1).
The key to the solution to this problem was presented in Meeting 7 (appendix C). We should have an intermediate class that sat in between the Front-end and the Back-end, and handled message passing accordingly.
9.3.2. An Intermediate Class
A class which handled messages passed from Front-end to Back-end would obviously need to be globally available. It would not be enough to have a separate instance of a class in both the Front-end and the Back-end. The messages would simply not propagate to their destination. Messages from the Back-end destined for the Front-end would have no way of reaching the Front-end. Therefore an initialisation class would be need to accompany a message class. This initialision class would set up the message area and then start the backend and the Front-end from the same class. The following diagram makes this structure clearer:
Figure 9.3.2. Implementation using middle and an initialisation class
Now both the Front-end and Back-end have access to the same global Message Area. The idea is much like a pigeon hole system. They can leave messages there with 'labels' on, and if anyone expects a message with that label they can remove it from the hole leaving it free for another message. The implementation is easily extended by allowing messages to imply that data is also ready for collection at a certain point. However this data must be stored within the MessageArea class and collected by the receiving class.
9.4. Message Area
9.4.1. Initial Requirements
The Message Area at its most basic must provide a mechanism to deposit a message and to collect a message. It must have at least two fields, one containing the message, and one containing any accompanying data.
9.4.2. Naive Implementation
The first implementation provides 4 methods, the put method has been overridden to allow a single message to be placed or a message and data to be placed.
public class MessageArea {
private String message;
private String strData;
private boolean available=false;
public void put(String theMessage) {
message = theMessage;
available=true;
}
public void put(String theMessage, String theData) {
message = theMessage;
strData = theData;
available=true;
}
public String get() {
return theMessage;
}
public String get() {
return theMessage;
}
}//end class
The MessageArea was used simply as in the code below. The code deals with card selection, where the backend must wait for a card to be selected by the user. This involves a Front-end -> Back-end message pass, with String data (the selected card).
Back-end Front-end
The backend needs a card before it The Front-end will only give a card
can continue when it receives one.
// Backend waits for a card // Front-end receives a user click on a card
String action = theMessageArea.get();
theMessageArea.put("CardSelected, theCard);
while (!action.equals("CardSelected")){ // continue running the GUI
action = theMessageArea.get();
}
theCard = theMessageArea.getData();
This implementation seemed to work, however in testing there were two serious issues. Firstly CPU resources were almost completely consumed and the GUI became very unresponsive and secondly and most worryingly if two cards were clicked in succession then only the most recent card would propagate through to the backend. Things got worse when we tried passing other messages as recent messages would overwrite earlier ones, even if they had not been collected. This led to events being "lost" and the software proved to be increasingly unreliable. The slowdown was caused by the Back-end constantly looping until it got the value it required. This approach is known a spinlock. ‘Spinlocks’ relentlessly poll a variable until some condition is met, thereby using up all the processor resources. Something needed to be done to improve the situation.
9.4.3. Producer - Consumer Analogy
Recent lectures on ‘Concurrency (G52CON)’ proposed a Producer-Consumer example where two threads operate on a shared buffer. One thread produced data and the other collected (or consumed) it. The parallels of this to the Message Area were obvious and more importantly the example provided solutions to problems experienced with our naive implementation. The code below is taken from the Sun Java tutorial. It can be found at
It provides a Producer-Consumer implementation in Java by using threads.
public class CubbyHole {
private int contents;
private boolean available = false;
public synchronized int get() {
while (available == false) {
try {
// wait for Producer to put value
wait();
} catch (InterruptedException e) {
}
available = false;
// notify Producer that value has been retrieved
notifyAll();
return contents;
}
public synchronized void put(int value) {
while (available == true) {
try {
// wait for Consumer to get value
wait();
} catch (InterruptedException e) {
}
contents = value;
available = true;
// notify Consumer that value has been set
notifyAll();
}
}// end class
public class ProducerConsumerTest {
public static void main(String[] args) {
// create shared cubbyhole
CubbyHole c = new CubbyHole();
// create the consumer and the producer
Producer p1 = new Producer(c, 1);
Consumer c1 = new Consumer(c, 1);
// start the threads
p1.start();
c1.start();
}
}// end class
The tutorial uses a Producer and a Consumer which operate on integers. The integers are stored in a “Cubby Hole”. The implementation starts the Producer and Consumer as two separate threads each accessing a shared CubbyHole object. The CubbyHole is passed to each thread as an argument to their constructors.
The get() and put() methods in the CubbyHole class exploit Java's concurrency methods to ensure three things:
1) Data is not lost by two methods accessing concurrently (synchronized keyword),
2) Data is not overwritten until it has been taken (boolean variable),
3) If no data is available the method goes to 'sleep' (wait() and notifyAll() ).
These three improvements address all the issues, which plagued the naive implementation. Points 1 and 2 address the issues relating to lost events and point 3 ensures that if we are waiting for an action to occur the CPU is not hogged by a ‘spinlock’ algorithm. By adapting this CubbyHole class we can turn it in to a thread safe version of MessageArea.
9.4.4. Building the MessageArea
There are 3 ways we will want to leave data in the MessageArea. We may want to leave an action to signal an event has taken place ( a button press). We may want to leave an action and one piece of data (placing the button) or an action and two pieces of data (adding a player). These have all been implemented in the MessageArea class by overriding the put method with the appropriate number of arguments. The fourth put method accounts for the fact that we can leave integer data instead of a string with an action.
The ‘get’ method is implemented as getAction(). This method returns an action if present or waits patiently otherwise. The boolean argument passed to the method flags whether the message should be removed, for most of the time this should be the case. When we have an action successfully returned from the MessageArea we would know if the message has any accompanying data. If so we can call getMessageData() or getIntegerData() to return any relevant information. These two methods are currently unsafe, as in theory the data concerning the message could be changed before being returned to the calling method. However the implementation is currently working and this can be addressed with further development.
9.4.5. The Implementation
Upon implementation it was clear in testing that even with this new thread-safe MessageArea, some messages were still being lost before they were acted upon. This turned out to because messages were being passed in two directions as shown in 3.1. Messages from the Front-end to the Back-end were not mutually exclusive to messages from the Back-end to the Front-end. A player can cause an event in the GUI at the same time as the Backend decides to send a message to the GUI. Whatever message ends up in the MessageArea is random, but only one will succeed and the other will be lost. The current MessageArea provided no way to queue these events in the manner of most industrial message pumps.
The simple solution was to provide two separate instances of MessageArea, one for each direction of message travel. The message would be placed in the appropriate store depending on the destination. This provides an effective method of two-way communication between the two classes. The two areas are created in the initialisation class Poker.
theExchange: handles messages travelling from the GUI to the Back-end
Such messages include:
AddPlayer: The user has signalled he would like to add a player
ButtonPlaced: The user has placed the button.
theGUIExchange: handles messages travelling from the Backend to the GUI
Such messages include:
ChangePlayer: The backend has moved the active player. The GUI should update
PlayerFolded: The player is no longer in the game. The GUI should show this.
theGUIExchange message handler is contained entirely in the initialisation class and is located after the Back-end and from end threads have been started. theExchange messages are handled through out the code.
9.4.6. The Messages
9.4.6.1 Message Syntax
During the course of development messages were added to the system as and when they were needed, using the most suitable and descriptive name. The design of the message area required the message to be contained in a String object. Usually a signal message indicating that some event has taken place will be of the structure:
NounVerb
e.g.
“ButtonPlaced”
A message requesting an action to be formed will be of the form:
VerbNoun
e.g.
“ResetGame”
There is no visible indication, with the message alone about what data should accompany the message. There could be no data, an integer, a String or both data types. It is up to the logic of the message recipient to decide what data is implied by the message.
This is, of course a very primitive system, and professional message pumps make extensive use of constants and other concurrency primitives for the purpose of signalling.
9.4.6.2 Implemented messages
Below follows a comprehensive overview of the messages found in the code, what they indicate and what data they deposit or need.
9.4.6.2.1 Messages to the Front-end
These messages will always been found in the MessageArea object called theGUIExchange.
|Message |Data |Receiving Class |Sending Class |
| |
|ChangePlayer | |Poker |GameInfo |
|Tells the GUI to highlight a player. This means that a player’s name changes colour to indicate their turn. |
| |
|GetCard | |Poker |PPDeck |
|Tells the GUI to inform the user that the back-end requires a card. |
| |
|PlayerFolded |int |Poker |Dealer |
|Tells the GUI to grey out a player and no longer show the player as active. |
| |
|DisplayCard | |Poker |PPDeck |
|Tells the GUI to display a certain card image on the board. |
| |
|ResetTable | |Poker |Misc.PokerGUI |
|Tells the GUI to clear all card images from the board and reset all players. Also signals the dealer to stop and reset. |
| |
|EndGame | |Poker |Backend |
|Signals the end of the game. |
9.4.6.2.2 Messages to the Back-end
These messages will always been found in the MessageArea object called theExchange.
|Message |Data |Receiving Class |Sending Class |
| |
|AllPlayersAdded | |Back-end |PokerGUI |
|The user clicks the confirm button after all players have been added. This signals the Backend to add the players for real. |
| |
|ButtonPlaced | |Back-end |PokerGUI |
|The user has decided on a place for the button. Set the button in the backend. |
| |
|PlaceButton |int |Back-end |PlayerPane |
|The user is placing the button but hasn’t confirmed a position |
| |
|AddPlayer |int, String |Back-end |PlayerPane |
|The user has finalised a player name (by pressing enter). This player must be added to the Dealer. |
| |
|ResetGame | |Back-end |Misc.PPDeck |
|Stop all actions and reset the dealer. |
10. Division of Labour
10.1. Meetings
The group had a regular meeting during the course to organise the project activities. There were two meeting; one was a formal meeting as it was with our supervisor Graham Kendall at it took place on Monday afternoon. During the meeting the group discussed the problems that they were facing while developing the program, last weeks progress and the tasks that each member in the group will do. These meetings usually would not last for more than 30 minutes.
In each meeting we had a chairperson and a secretary. The chairperson will lead the group during the meeting, discuss previous action, unfinished action and ensure that each member in the group have a task to do for the following week. On the other hand, the secretary takes notes during the meeting about any important notes that have been discussed during the meeting. After taking the notes and the attendance the secretary compiles the notes and adds them to the group web site where it will be easy for the members of the group to refer to. These positions were not assigned to a specific member as each one took turn in it.
The second meeting that we had was the informal meeting that took place every Thursday. In these meeting the group used to meet to discuss the progress that have been done in the project and prepare them self for the following meeting with the supervisor
Examples for some of the important minutes that were taken during some of the formal meetings: see Appendix C.
10.2. Web Site
In order to keep in track with the minutes that was being taken by the secretary every week in the formal meetings. The group decided to have a web site where all the minutes will be in, group member’s details i.e. email addresses to allow quicker interaction between the members and details of other means of communication i.e. telephone number, other email addresses and so on. Mike volunteered to create the web site,. It was the secretary’s job to update the web site with the most recent minutes after each formal meeting, adding the most important aspects of the meeting. This allowed each member to have a full idea about the project progress and the tasks that were assigned to out side the meetings.
The group also had a specific email address (gp-gxk2@cs.nott.ac.uk) that was given by the school of Computer Science to allow members to communicate with each other at anytime; also the supervisor is also on this mailing list.
10.3. Time plan
The following is the time plan that we kept and updated throughout the project. This time plan allows us to meet deadlines that we kept for our selves and details the official deadlines and important dates set by the School (also holidays were taken in to account).
September:
25th - Start of the Autumn Semester.
26th - Last week of September, we met up as a group for the first time. Informal introductions were made with group members and our supervisor. General discussion took place of the project title.
October:
2nd - first meeting for the group with the supervisor.
9th - first formal meeting.
10th to 19th - learned how to play Poker on and amongst ourselves, read the journals that were given by Graham Kendall,
22nd to 31st - did some research about Poker classes and started designing the initial GUI.
November:
1st to 9th - split up in to two groups, one group work on the GUI in Visual Basic, the other group started to work with the ‘Poki Classes’,
12th to 20th - start creating the GUI using Java, other group start work on the Back-end with some of the ‘Poki Classes’, start work on the Interim report,
21st to 30th - work on the Interim Individual and Group reports.
December
:
1st to 5th - complete the Interim Individual and Group reports,
5th - Interim Individual and Group reports deadline,
6th to 14th - organised the time plan for the project, continuing to work on the back-end and the front-end.
15th - end of term.
EXAMINATION PERIOD
January:
28th - start of the 2nd semester,
29th - first group meeting in this semester.
February:
1st to 28th - continue to work on the GUI and the actual program.
March:
1st to 22nd - combining the Front-End and the Back-End,
23rd - last day of the spring term.
April:
23rd - start of the summer term,
24th to 30th - testing the program,
24th to 30th - prepare Final Group Report and Final Individual Report,
24th to 30th - preparing for the Open day and the Presentation
May:
1st - Final Group Report and Final Individual Report deadline,
2nd to 7th - Working on the Presentation and Open day,
8th - Open day,
9th - preparing for the Presentation.
10th - Presentation
PROJECT COMPLETE
11. Evaluation
11.1 Limitations of the Software
The development cycle of the software was not optimal. There was very little planning and a lot of programming. It was decided to try and adapt the ‘University of Alberta Poki classes’ to our needs, and although these classes have proved extremely useful they were never designed to be used in the way that we have implemented them. Therefore there came a stage where it seemed easier to carry on despite the potential of lots of messy code than turn back and start afresh. The result is a program that work reasonably well but which has a number of limitations.
11.1.1 Limitations of the GUI
The GUI was written in Java using the Swing API. This API is not known for its runtime speed, which leads us to out first problem. The loading time for the software is extremely long. This is mainly down to the relatively large background image, a 112Kb JPG.
The method of adding players to the games is admittedly somewhat cumbersome. Firstly Players must be added in order, working clockwise from the 7 o’clock position. This is certainly not the interface used at Paradise Poker, however this is one way that the back end’s idiosyncrasies really impose themselves on the GUI. The dealer adds the players as it receives them from the GUI, therefore adding a player further round the table before one earlier on would result in the order of the play becoming muddled.
The user must also be careful when and where they click, as the program is nowhere near as robust as it should be .For example, to add a player the user must click on the chair but not on the text. A click on the player’s label will have no affect. This is down to the chosen implementation of an ActionListener in Java, and it became too late in the project to change it. Also it is possible to click on cards in the CardWindow throughout the game, even when the backend is not expecting such an event. This results in cards being ‘lost’ from the deck as the CardWindow makes the icons invisible upon a mouse click. This means that that particular card is unable to be used for that round.
11.1.2. Limitations of the Message Area
The message Area is a very quick and dirty method of providing a communication ‘tunnel’ between back end and front end. As a result if an event occurs that we are not expecting, the system may become confused resulting in a serious program error. Since the backend is effectively synchronous this is not a problem, however the front end can generate messages at anytime. It is unknown, due to a limited testing period exactly how robust this system is. Through careful use however the program has performed surprisingly well.
11.1.3. Limitations of the AI
Unlike its namesake, there are some limitations with JessePlayer. The game engine does not implement bluffing. There is a degree of randomness provided by the probability triples, but it does not purposely go out to deceive its opponents. Implementing bluffing in software is difficult. Professional poker players rely on years of experience to make them able to fool opponents. Whilst a computer might have the World’s best poker face, it’s inability to read other players makes it vulnerable.
There is currently another problem that occurs under certain circumstances. If JessePlayer decides it’s hand is good enough to bet/raise, it will continue to raise until the betting limit of Paradise Poker is reached or all other players check. A knowledgeable opponent could exploit this if they have a very good hand by continuing to re-raise which would cause JessePlayer to lose more money that it might otherwise have done. This type of play is not too bad as most players on Paradise Poker play tables are loose. We are working on ways of fixing this bug and JessePlayer should not exhibit this behaviour on the open day.
11.2. Overall Successes
The competition is in the bag!!!
12. Bibliography
1 Billings D, Pena L, Schaeffer J, and Duane Szafron D. Using Probabilistic Knowledge and Simulation to Play Poker, AAAI, pp. 697-703, 1999
2 Billings D, Papp D, Schaeffer J, and Szafron D, Opponent Modelling in Poker, AAAI'98, pp. 493-499, 1998
3
4
5
6. Billings D, Pena L, Schaeffer J and Szafron D. Learning to Play Strong Poker, ICML-99 Workshop Machine Learning in Game Playing, J. Stefan Institute, Slovenia
7. Billings D, Davidson A, Schaeffer J, and Szafron D, The Challenge of Poker, Artificial Intelligence Journal, 2000
8 BillingsD, Papp D, Pena L, Schaeffer J, and Szafron D. Using Selective-Sampling Simulations in Poker , AAAI Spring Symposium on Search with Imperfect Information, pp. 13-18, 1999.
9. Billings D, Papp D, Schaeffer J, and Szafron D. Poker as a Test bed for Machine Intelligence Research, AI'98, in Advances in Artificial Intelligence (Robert Mercer and Eric Neufeld, eds), Spinger Verlag, pp. 228-238, 1998
10 Java Examples in a Nutshell, David Flanagan,1997
11
12
Other journals that were looked at:
• Billings D, Computer Poker. M.Sc. research essay, 1995.
• Davidson A, Billings D, Schaeffer J, and Szafron D., Improved Opponent Modelling in Poker. Proceedings of International, Conference on Artificial Intelligence (ICAI'2000), 2000.
• Denis R., Dealing with Imperfect Information in Poker,1998,
• Korb, K. B., Nicholson, A. E., and Jitnah, N. 1999. Bayesian poker. In Proc. 15th Conference on Uncertainty in Articial Intelligence (1999). Morgan Kaufman.
• Schaeffer J. The Role of Games in Understanding Computational Intelligence, IEEE Intelligent Systems, pp. 10-11, November/December, 1999.
• Schaeffer J. The Games Computers (and People) Play, Advances in Computers 50, Marvin Zelkowitz editor, Academic Press, pp. 189-266, 2000.
• Schaeffer J. Search Ideas in Chinook , Games in AI Research, Jaap van den Herik and Hiroyuki Iida (editors), pp. 19-30, 2000.
• guide3.html
• html-English/quicktour.html
• index.html?Example_Session=063ee49c1a5f2aaabe33201ce97b3bad
•
• texasholdem.html
• poker/dictionary.html
• pokerglossary.html
• cs.cmu.edu/afs/cs/user/mummert/public/www/ircbot.html
•
•
•
• cs.ualberta.ca/~games/poker/
• texas-holdem.php
•
•
• How to Play Poker and Win written by Brian McNally with John Thompson and Barry Mcilroy
• “Late Night Poker” on Channel 4
Appendix
Appendix Ai
[pic]
Appendix Aii
[pic]
Appendix Bi
First successful implementation of the Poki classes
import poker.*;
public class PokiBackEnd {
public static void main(String[] args) {
// instantiate new players
Player playerRandom = new RandomPlayer(0.1,0.5,0.4);
Player playerHuman = new poker.ai.player.HumanPlayer();
// set up the main Dealer class
Dealer myDealer = new Dealer();
// add player to the game
myDealer.addPlayer("Random", playerRandom);
myDealer.addPlayer("Human", playerHuman);
//start the game
myDealer.playGame();
}
}
Appendix Bii - The engine at 28/11/2001 fully using the ‘Poki’ Classes
import poker.*;
public class PokerEngine {
public static void main(String[] args) {
// establish how many players besides ourselves
System.out.println("How many Paradise Poker players?");
int numPlayers = UserInput.readInt();
// class to represent the opposition
Player ppPlayer = new poker.ai.player.HumanPlayer();
// class to represent ourselves
// a balanced player: 0.3=fold, 0.4=check, 0.3=raise
Player ourPlayer = new poker.ai.player.RandomPlayer(0.3,0.4,0.3);
Dealer myDealer = new Dealer();
// We add ourselves to the game
myDealer.addPlayer("Jesse", ourPlayer);
// then everyone else
for (int i=1; i Jesse 1
> James 1
-----------------------
Enter PP Card:
jc
Rank: j Suite: c
Extracting Jc from the deck!
Enter PP Card:
3h
Rank: 3 Suite: h
Extracting 3h from the deck!
Enter PP Card:
ac
Rank: a Suite: c
Extracting Ac from the deck!
Enter PP Card:
6h
Rank: 6 Suite: h
Extracting 6h from the deck!
James: Your Hole Cards Are: Ac 6h
Jesse blinds $1
James blinds $3
Jesse calls $2
Your Action James [Ac-6h] : c
James checks
Enter PP Card:
jd
Rank: j Suite: d
Extracting Jd from the deck!
Enter PP Card:
9h
Rank: 9 Suite: h
Extracting 9h from the deck!
Enter PP Card:
ts
Rank: t Suite: s
Extracting Ts from the deck!
************************************
FLOP: Jd 9h Ts $6
************************************
Your Action James [Ac-6h] : c
James checks
Jesse checks
Enter PP Card:
7s
Rank: 7 Suite: s
Extracting 7s from the deck!
************************************
TURN: Jd 9h Ts 7s $6
************************************
Your Action James [Ac-6h] : c
James checks
Jesse checks
Enter PP Card:
6d
Rank: 6 Suite: d
Extracting 6d from the deck!
************************************
RIVR: Jd 9h Ts 7s 6d $6
************************************
Your Action James [Ac-6h] : c
James checks
Jesse checks
Jesse: Jc 3h
James: Ac 6h
Jesse wins $6 with a Pair of Jacks
Appendix C – Meeting Minutes
Monday 4th February 2002/ 14:00 – 14:30
Chairperson: Tim
Secretary: Wafa
Present:
Graham Kendall, Tim, Michael, Viral, Kevin, Wafa
Absent:
Keeran
Points Discussed:
• The report result was given. Graham discussed some points that need to be improved on the group report. References should be added!!
• Tim and Mike discussed the back-end. Graham also raised some problems that could arise while writing the back-end that could result for the whole thing to crash.
• He also mentioned the importance to play a legal game.
• The members also discussed how are they planning to combine the back-end with the front-end. Graham suggested creating a class that deals with both ends.
• The open day was also covered in this meeting as well as the presentation and Who will present.
• The two teams should discuss the way that they are going to play the game on the open day to avoid any surprises.
Actions:
• Viral, Wafa : writing the group report and starting to work on the open day
• Tim, Mike: continue to work on the Back-end
• Kevin: continue to work on the GUI
Next Meeting:
• Informal Meeting: Thursday, 7th Feb at 13:00 pm in the exchange
• Formal Meeting: Monday, 11th Feb at 14:00
o Chairperson: Viral
o Secretary: Kevin
Guys 40 Days left so its time to work!!
-----------------------
[1] The first two players’ are forced to put a fixed size of bet to start the game.
-----------------------
Main Area
Card Window
Dealer
Box for players name and buttons for each player’s actions.
Main table area
Text box for any comments from dealer or the players
Action buttons to Leave Table or exit game completely
instantiates
Player
instantiates
instantiates
instantiates
instantiates
Dealer
Backend
Middle Class
MessageArea
PokerGUI
Initialisation Class
Poker
public class RandomPlayer extends Player {
GameInfo gi;
ProbTriple pt;
public RandomPlayer(double f, double c, double r) {
pt = new ProbTriple(f,c,r);
}
public void init(Preferences playerPrefs) {
pt = new ProbTriple(playerPrefs.getIntPreference("FOLD"),
playerPrefs.getIntPreference("CALL"),
playerPrefs.getIntPreference("RAISE"));
}
public void newGame(GameInfo gInfo, Card c1, Card c2, int ID) {
gi = gInfo;
}
public void update(int action, int code) {}
public int action() {
return pt.select(java.lang.Math.random());
}
}
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- university of minnesota college of education
- university of minnesota school of social work
- wharton school of the university of pennsylvania
- cost of university of scranton
- university of minnesota school of education
- university of scranton cost of attendance
- university of south florida college of medicine
- university of minnesota masters of social work
- ecampus of university of phoenix
- university of minnesota college of continuing education
- university of illinois college of nursing
- university of north texas college of nursing