PERFORMANCE ANALYSIS: USING SIMULATIONS TO OBSERVE COLLECTIVE BEHAVIOR

Since the early 1920s, simulations have been used as resource for the strategic planning of state and non-state organizations. It has been applied as didactic and analytical tools to interpret actors’ behavior in decision-making arenas, by means of the development of a theoretical-methodological apparatus capable of analyzing multiple variables in an increasingly complex environment. When thinking about the didactic and strategic needs of the military institutions, under the Simulation and Scenarios Laboratory (Brazilian Naval War College), we were dedicated to developing a method for analyzing the performance of players in competitive and cooperative games. Thus, throughout this article, we will present the constitutive phases of the method called Performance Analysis, as well as the results achieved by means of its application in the games developed by the above mentioned laboratory.


INTRODUÇÃO
This work is about years of experiments in developing the expertise to model simulations on conjectures built in competitive and cooperative designs. Observing crisis cases was progressively used within the Brazilian national security institutions (both defense and security) in benefit of agent behavior analysis, either for training rules of engagement and legal marks or trying to develop new forms of arrangement and refining performance.
The instruments associated with wargaming were the main window of opportunity to expand knowledge into crisis management and internationalization process, where acceding countries were making their military representation progressively more attached to diplomatic instances. Inside a foreign relations framework, evidenced by its diplomatic skills, Brazil had also advanced its choices and strategic arc considerably. As a result, crisis management models were matured as to permit actors to enhance skills that were not usually part of the career development, advancing performance on international representation, where scenes were conceptually competitive.
When the World Cup and the Olympic Games were being planned as part of international, federal and federative institutions' provisions, there was the need of adapting the models developed to make possible Brazilian actors' better achievements in the international sphere, to those sights markedly cooperative and interagency. This was due to the fact that, at the height of 2012, the cooperative instruments and protocols were far too precarious to brook any delay in tackling this issue, as preparedness was taking place in various grades inside the Brazilian security and defense institutions.
Moreover, there was a lack of studies in between practice and academia, especially considering long-term planning. Given the recent nature of the International Relations area in Brazil, progress on methods was rare, and upholding initiatives that were constituted firmly inside defense and security institutions with theoretical support seemed to be a way to contribute to the field. Also, although tools were fostered because of the demands laid down by the defense institutions on their preparation, there was not a visible memory of those contributions over time.
Therefore, collaboration was tested in a double way: on the one hand, in simulations where international actors should cooperate in benefit of the group -as Rawls' overlapping consensus (Rawls, 1985:251); and on the other, promoting protocols progress in putting different domestic agencies and actors' mindsets in the same arena. Such uses had permitted researchers and practitioners make improvements on the manner control groups should act, including the subject matter experts got together to the task. Accordingly, observation in a modernized set of criteria was the Achilles' heel in the simulations and gaming modeling context.
Paying close attention to the needs of the state for the coming years, Performance Analysis Method (PAM) was developed by this papers' authors first to examine the core procedures on planning decisions and negotiating them as to influence the final output of a given situation. With this regard, competitive scenarios were provoking actors to observe the different domains and interests that eventually one could not control. Hidden or biased information would be the premise of the bounded rationality that moves actors towards internationalization, despite its limits. Conversely, whenever simulations would be prepared to enhance cooperation and the standardization of processes, information sharing and transparency would be the guidance.
In this way, PAM is a method arisen under the auspices of the Laboratory of Simulations and Scenarios of the Brazilian Naval War College, one of the leading players thinking about defense diplomacy, internationalization and the improvement of interagency cooperation in defense and security matters (Winger, 2014). Leading the project, the authors modeled a group of simulations during the past years, with different scopes and actors, testing various outbursts during processes of the projected simulations. The method gave reason to the creation of a software program after it was developed in tables that would permit inputs of data and results, organizing some of the measurement factors proposed.
The researchers' objective with the software was not to show pure results without considering the analytical process behind each one of the phases of the simulation. Thereby, the division of the simulation analysis in phases was to assure that researchers could afford to have a disciplined procedure with different roles. First, the variables were considered from the scope of the simulation, as competitive or cooperative. Second, the method was built to permit the analysis of the strategy's coherence of each actor and the policy feasibility in terms of the subject he/she represents. Given that relation, the variables used in this phase of analysis compose the dimension coherence that is observed by grouping both results in accordance. Finally exposed in a graphic representation, the data is contrasted to the general influence of the actor based on the final output (a resolution, for instance).
Besides the developments on the method to achieve the analysis of the behavior of the actors whenever there are conflicting interests, cooperative simulations would have differed from the competitive ones, the common objective of the representatives. With this in mind, an adaptation to the tool was made, substituting the collection of discourses during the competitive simulation for establishing preconditions and variables to be analyzed commonly among actors in the case of cooperative scenes. This process shows that it is possible to achieve better results in cooperative scenarios once coordinators know the best practices to expect and transform them into joint variables to be observed. Both ways of using the tool (which is available online at crisisgame.org, in its 2.0 version) consider distinct groups of detecting behaviors, establishing expected behaviors or analyzing them (SME's, Subject Matters Experts).
This work will present processes and findings of the method during the last 5 (five) years, in light of the discussions presented on international relations and simulations methods for observing actors and institutional behavior. As this a long-term project, results are still in progress and under development because, as an analytical tool, it has space continuously to better fit into the constantly changing expectations and objectives. Our proposal is, after presenting the parallels to the literature and marking the phases that constitute the method, to go through an analysis of the different mechanisms that have market both competitive and cooperative scenarios, giving birth to a comparison of the tool and possible benefits to achieve the final proposal of observing tendencies and possibly measuring better consequences of a collective agenda.

BACKGROUND
Since the twentieth century, methods of simulation of critical periods or war conjunctures have been the object of interest of military institutions linked to strategic planning or tactical training. Thus, a series of models emerged inside war planning and academic institutions, focused on predicting states' behavior. Whether it was based on concerted action by states or on the training of maneuvers involving the right combination of movements, the presence of increasingly complex decision-making processes took shape. In effect, the attempt to rationalize these movements gained new contours, fundamentally, after World War II and the emergence of another model of confrontation during the Cold War (Perla, McGrady, 2011;Caffrey, 2000).
Currently, a variety of simulation models is used, which includes educational and/ or analytical purposes. According to Thomas (2002), in the classroom environment, simulations can serve as an effective teaching tool in provoking behavior conditions from rational choice to intuitive decision making. Regarding the international relations arena, there can be in place relative versus absolute gains or the anarchic nature of the international system versus the tendency towards cooperation. For Streufert et al. (1965), games can be used to simulate organizational environments with the objective of measuring the variables that contribute to the success of specific activities. Also, it can be used in benefit of the analysis of variables in a complex context of interaction between individuals, thus assuming, a more rational approach. The United Nations model, for example, has been used since the 1920s, when the first simulations on the League of Nations took place. However, as established today, the model was only proposed in 1952 by the Model United Nations (MUN) of Berkeley in the United States. The purpose of the MUN is to facilitate the experimentation of dispute settlement and negotiation involving state actors, international governmental and nongovernmental organizations, and it has been developed for several purposes, among which the promotion of a culture of peace (Rojas Veras et al., 2006).
Progressively, the culture of peace has gained relevance not only in the United Nations (UN) but also on other inter-state institutions' agendas, and it has advanced based on deepening diplomatic and para-diplomatic institutional mechanisms (Pereira, 2005). As long as the culture of peace is spreading, diplomacy is gaining a new proportion, as there is a tendency of distributing the internationalization attribution among states' agencies and ministries. This configures a renewed mediation concerning states' representation, side by side with the foreign ministries relations' offices.
At a time when diplomacy has gained a greater magnitude, defense diplomacy has also been accentuated, in the light of progress on cooperation schemes in the sector (Winger, 2014). Therefore, if negotiation processes are more likely to be used than before, for defense diplomacy it is important to note the way simulations can permit advances on: 1) Crisis simulations and wargaming as methods of learning, testing and analyzing.
2) The possibility of observing and improving behaviors based on collective dynamics.
3) The possibility of obtaining short-term scenarios based on the conjuncture created in simulating fictitious characters and scenes, however, with a high incidence of reality.
War Games have been used as tools in the Brazilian Naval War College (Escola de Guerra Naval -EGN) since the beginning of the 20th century. As part of a tradition of strategic military studies, the terminology "game" signals the performative understanding of a general situation in a simplified and didactically organized way. Along these lines, the games permitted EGN both to train officers and create a unique database of analyses that could inspire future strategic decisions.
From this perspective, the Brazilian Naval War College served as a laboratory for the formulation of cooperative and competitive games, fundamentally after the launch of the naval war simulator, developed between the 2000's and 2010's. Simulations, since the board games era, once applied to military and strategic environments of large and medium powers, enables us to observe the tactical/operational application of the training aspects. Through the study of game theories, the universe of simulations was expanded to the uses and models in which decision processes were also analyzed in course. Thus, the simulations have become technically better elaborated, with more developed capacities concerning the phases, the targets, the observation processes, the metrics, and the variables to be analyzed.
Along with the greater ability to handle complexity and administrative detail came a potential for loss of "transparency"-awareness by players of a game's underlying assumptions-and a temptation to add "realistic features" to games, because it was technologically very easy to do so, without thinking much about whether the additions added to or detracted from the games' underlying purposes. The push for added complexity rarely came from the people who thought games were a good way to test concepts or plans. Rather, it originated mainly from the technical community of analysts and gamers. There is now a divide between an increasingly specialized community of gamers and modelers on the one hand and policy makers on the other; this divide is greater today than it was in the 1970s. Gamers have to market their capabilities the way any business does. There is nothing wrong with this, per se. However, experience indicates that this marketing, and much of gaming's development over the past twenty years, has been aimed at other gamers rather than the policy-making community. It has been aimed even less at casting light on new challenges to U.S. security management, challenges that barely existed twenty years ago. (BRACKEN, SHUBIK, p.48, 2001) To meet this perspective, the Laboratory of Simulations and Scenarios (LSC) was created in 2012, as part of the Brazilian Naval War College, side by side with the Center for Political-Strategic Studies. In close collaboration with the EGN War Game Center and with the support of civilian and military experts, the simulations have been the object of study to enhance defense decision-making processes in pre-war games held for military training or civilian simulations also built with the same purposes. Since then, the Laboratory has been divided into three main specialties, namely, simulation models, performance analysis of the actors, and scenario building.
Thus, the games are organized by their main target: (1) analytical and/or (2) educational. In the case of games of an analytical type, the central objective is usually not delineated, since the consequences of the simulation are the most important results. Therefore, results may not be foreseen until the simulation is concluded. For this, it is important to fully engage actors in the simulacrum of this reality, which means the performance of the actors may be considered high at the end of the game, as to make possible the constitution of the desired effect.
Otherwise, in simulations of an educational nature, the objective will generally be constituted before, in order to allow the actors to be tested or test institutional systems and subsystems, such as rules and situations in which they need training and learning. It is clear that the analytical element of the game is concentrated in the case to be studied and possible consequences, however, it is not only represented by this. It is also possible to characterize the analysis by individual behavior as a specific goal. In this case, the reactions of each actor are observed, not the consequences of the negotiation.
As a derivative of the war games, crisis games continued to involve a number of individual stakeholders in the midst of the challenge of solving problems at the lowest cost possible, where the spectrum of the culture of peace was crucial. Thus, the games consisted of negotiations, cooperation, and diplomacy, but competition also marked their development for educational purposes. For this reason, we agreed to classify games into those of a competitive origin and those of a cooperative origin. This origin is marked by the need to cooperate or not, based on the type of subject. If the game is of a competitive origin (1), although actors are compelled to cooperate, there is no guarantee that they will decide for cooperation, unless negotiation around cooperation seems rationally convenient to them. Otherwise, when the game is cooperative (2) in its origin, in particular, they are actors of a state or organization that, aiming at the fulfillment of a common objective, should create conditions for the coordination of ideas and methods for the implementation of a predefined general objective.
We are going to describe the Performance Analysis Method (PAM), first, using cases that represent competitive games and, subsequently, the cases that represent the cooperative games. Finally, we are going to address the results and their consequences to the observation of actors' behaviors in complex scenes.

COMPETITIVE SIMULATIONS AND PERFORMANCE ANALYSIS METHOD
The Model United Nations (MUN-LSC-EGN) is an activity developed annually by the Laboratory of Simulations and Scenarios and counts on the participation of undergraduates and graduates generally from International Law, Political Science/International Relations, Economics and History, as well as students enrolled in the Brazilian military institutions' academies. For many years the MUN was used as the testing scene of the PAM as to improve it, permitting the achievement of the results using tools for settling controversies. The model was also imbibed into pre-war game scenarios and other simulations which involved conflict resolution and competitive disputes.
The choice of the topic to be simulated is usually linked to the demands of the Brazilian Naval War College, related to maritime and international security issues. The Model United Nations has been applied by different institutions categorizing a simulation with themes related to diplomatic mediation, manifesting a range of discussions that are similar to the decision-making process performed at the UN. Indeed, the educational purpose of those models (MUN) has been approached by many scholars, guaranteeing that procedures and frameworks can be better observed and imbibed (Kusma, 1998;Wilkerson, Fruland, 2006;Matzner, Herrenbrück, 2017).
Once the participants receive a statement of the case to be resolved and the actor's pre-conditions, the game is initiated. Thus, the participants simulate the debates addressing the strategic points, elaborating studies on the international ties, historical or political path, and the economic paradigms undertaken of the represented countries, as a way of acting through cooperation or competition and reaching their maximum objectives. Since then, formal and informal diplomatic statements have been carried out by the delegates, generating a sequence of discussions and possible agreements, with the purpose of reaching the structure of a resolution signed by the majority, proposing all the questions dimensioned throughout the simulation.
However, the delegations' performance was subject to precarious and subjective evaluations. Most parts of analysis were registered by the formulation or not of a resolution and by the conduct of the participants of the game through the rules. This method of simulating did not itself count on a deeper analytical framework in order to know the main elements that lead to a strong performance, with a sufficient knowledge apparatus, beyond the rules, but by the coherent set of strategies adopted from: 1. analysis of the delegations' discourse; 2. the coherence of the objectives outlined; 3. compatibility with the external policies of the countries represented; 4. agreements and resolutions.
In order to serve the purpose that was established, the simulations have been the object for the development of the methodological tool presented herein to analyze the behavior of the actors. The simulation model has adapted some of the rules used in other United Nations models but is based on commonly-held regulations for dialogue, proposal submission and voting. Having absorbed elements traditionally linked to the crisis and war games' models and having restricted rules of debate and proposal, the MUN, as a competitive game model, was the first one from which the performance analysis project was derived. At this moment, we are going to explain the method for the effect of the performance analysis in competitive simulation models. Thus, in the following lines, we present the rules and stages that guide the proposed methodological structure, considering the variables measured and evaluated throughout the process.
The game is initiated after the previous reading by the involved actors of some basic documents: the case to be debated (case study); the rules of the debate (rules guide) and the mini-max document. The latter is the document whose model is available to registered players to define their maximum and minimum goals, their potential allies, and their potential opponents. This document is fundamental in competitive games so that the planning of the actors can be observed together with the whole simulation process.
Once the Assembly is established -where debates are conducted -, delegations present themselves so that they all make an initial statement with the central ideas they wish to point out. From that point on, debate sessions will be moderated by the governing staff of the game (debate directors), and sessions will be interspersed with non-moderated debates, eventually approved in the Assembly, of smaller intervals of time. Throughout this process, it is possible for players -usually organized in delegations -to be represented by more than one member so that bilateral and multilateral meetings, suggested by the actors themselves and in parallel to the Assembly, can be authorized. Delegates may act as advisors or speakers to their respective delegations and, once outside of the Assembly, they may act particularly concerning their strategic partnerships through diplomatic courier or direct negotiation.
A specific software to manage the structure was built to organize the game, actors' roles, analysis, and results, which are part of the Performance Analysis proposed. Most parts of the games are played having Portuguese as the main language -cases shown here are in Portuguese. Delegations should be assigned to the system one by one, based on the following fields: name of the delegation (Actor Name); representative and institutional affiliation (Actor Description); Password (to be defined by the Control Group and provided to users). Once actors are in the system, the Control Group (GRUCON) will initiate the data collection process for the Performance Analysis, a process that is divided into stages, as described below. The case in question was distributed in advance by the Control Group to players and was aimed at solving problems in the Gulf of Guinea, and the forum chosen for the simulation was the United Nations General Assembly. The Rules Guide should also be provided prior to the simulation, for the early preparation of the actors, ensuring a higher chance of obtaining a simulacrum of reality. As in the example, the countries are marked by the central themes and positions they represent during the discourses and negotiations in which they participate, where Iran evokes what they call a possible new oil crisis, Australia stimulates cooperation among the members of the Assembly, and Lebanon proposes naval cooperation agreements to reinforce multinational presence in the Gulf of Guinea.
It is important to note that the POS can be extracted from the debates, but not only. Eventually, there are some experiences in which it is possible to extract hidden positions from the actors whenever the bilateral or multilateral relations are essential and are predominantly out of the arena. It may be important to consider the value of having more agents collecting the POS. The cost of having more than one agent to collect is about the unique parameter used when considering one agent collecting information. This may reduce the lack of understanding in the way agents organize and put data inside the system.

b)
The Compatibility of the POS regarding the represented actors: Once the Original Subject Positions (POS) are defined, the corresponding table is distributed to a number of experts, preferably external to GRUCON. The experts will be registered in the system according to the figure below: The experts will assess each of the original positions of the delegations according to the comparison between the position adopted and the actual (foreign) policy of the represented country/actor. In this way, the experts will assign a value between 1 and 5, on a Likert scale, where: 1 = very low compatibility 2 = low compatibility 3 = medium compatibility 4 = high compatibility 5 = very high compatibility The central objective of this phase of analysis is to observe the degree of correspondence of what the actors represent in the real world (or, in a previous given fictitious scene). If they are international state actors, the analysis will focus on their foreign policy, in relation to the positions observed and placed in the software by the agent from GRUCON. In this case, lower or higher compatibility of one or more POS (listed for each delegation), will be assigned by the experts according to the likelihood of the original (foreign) policy of this particular delegation.
Therefore, the result of this evaluation by the experts will express the joint effect of this analysis, in a proportionally distributed way, by means of the arithmetic average obtained among the values given (in between very high and very low compatibility). Experts are chosen particularly for each simulation and are not distributed based on their grading the level of their knowledge, as usually in their knowledge, as it is done in some sources of experts' analysis. Therefore, the expert assessment system requires proper and private login and password. Through the software integrated into the management system of the Control Group, the experts select the values one by one of the Original Subject Positions (POS) assigned by the agent previously (1 to 5).
The content to be evaluated must be compatible (feasible) with that offered by GRUCON as a real, when applicable, or fictitious actor. Hence, representatives of the delegations must behave in accordance with the characteristics available and studied to acquire more compatible POS from the experts' analysis.

c)
Conformity of the POS observed in the light of the Mini-Max document: The experts, at this stage, are invited to analyze the position paper, which was considered the Mini-Max Model, used in the Brazilian Naval War College in various models of war games (Correa, Andrade Flor, 2015). The Mini-Max document is composed in its reduced version of four fundamental elements: maximum objectives, minimum objectives, potential partners and enemies/barriers to be transposed. This document must be placed in the system by each delegation, whose login must also be provided by GRUCON.
The experts will evaluate the degree of coherence of the Mini-Max in relation to the set of Original Positions (POS) already validated in the previous phase. Once again, the experts will assign a value between 1 and 5, on a Likert scale, where: 1 = very low conformity 2 = low conformity 3 = medium conformity 4 = high conformity 5 = very high conformity It is important to highlight that the analysis model can be oriented both by the Compatibility factor and the Conformity factor (that are part of the same Dimension Coherence), as well as opposed to each other, according to the analytical objective. Thus, if the objective is focused on how efficient the delegations were, the Compatibility factor is added to the Coherence factor, as variables to be observed regarding a single Dimension, here called Coherence.

Dimension Coherence = (Compatibility/ Policy Feasibility + Conformity/Strategy) if
Influence is the other dimension.
These two factors (Compatibility and Conformity), combined here for competitive United Nations Model gaming effects, may be observed together (system average), as opposed to Dimension Influence, which should be observed in the next phase. This result may allow observing if there is a directly proportional relationship between the degree of Coherence (Compatibility and Conformity) and the degree of Influence among the delegations. The values established for the Coherence Dimension were 50% for each factor.
For the next phase, Influence, the proportion is 100%, since only one factor, Influence, composes this Dimension Influence. An important point regarding fictitious cases is the need to offer previous subsidies to the players, so that knowledge about themselves and other players is another element to be provoked during the game by GRUCON. This occurs when the objective is predominantly educational, and it is necessary to observe the variation between the level of information about each self and the other actors and the capacity to play with a higher level of rationality. The level of information is directly proportional to the capacity of making decisions when we deal with game theory (Smith, 1988Axelrod, 1997Hirshleifer, Rasmusen, 1989). Therefore, in fictitious cases, it is necessary that, even if the information is gradually offered to the players, the central elements are defined previously.
As an example, we can cite a table created in the fictitious game offered in the Master Course in Hemispheric Studies of the Inter-American Defense College (IADC, Washington, DC; 2014/2015), in which the Control Group managed a crisis game whose actors were components of a fictitious continent, for which parameters and metrics (maps and contents) were defined and distributed to the actors throughout the process. This game was done and approved using the Brazilian Naval War College methods and software. It should be noted that this model has changed over the years in competitive and, more recently, cooperative simulations. For this reason, the software was built to allow the control group (GRUCON) to establish the proportions and names (variables) for each Dimension analyzed, in accordance with the analytical purpose of the game. Currently, the model under consideration has two factors of equal proportion for the Dimension Coherence (Policy Feasibility and Compatibility), as well as one factor for the Dimension Influence (Results' Influence). Here is the flexibility of a model that must be in constant debate, testing, and adaptation.

d)
The analysis of the Influence: At this stage, the experts will evaluate, with increasing indexes from 1 to 5, the level of Influence of each delegation, when fulfilling its objectives in the Final Document, created by the delegations as a result of the game. The final document, as soon as it is available, must be entered into the system for the experts to analyze it. It is possible to make a complete analysis by recording the partial results and phases of the game. In this case, separate numbered documents will be analyzed at different times. Partial results may be available for delegations depending on the objectives of the game.
In this way, it is necessary to emphasize that the phases of analysis of these methods are interconnected and, therefore, willing to generate a result with credibility and confidence. The Dimension Influence will be observed through the final document if the delegations approve it, and experts should use previous steps to do the final analysis. It is, then, a feature of in this method that the steps are correlated, compared and articulated to provide the most unbiased result possible. Once the initial result is revealed and, as it is a qualitative method, a general analysis of the results and their implications is necessary for the resolution of the case study chosen at the time. At this stage, a report should be produced containing such considerations regarding the scenario proposed by the simulation.
In this way, after analyzing the data released in the system, it is possible to generate reports and charts that clarify the degree of influence and conformity of the foreign policy presented by each of the delegations. Finally, based on the statistical analysis produced by the system, it is possible to choose the delegation that most approached the expected results in the simulated environment. Traditionally, after the presentation of the reports to the actors of the simulations, an Honorable Mention is given to the delegation that presented the highest degree of influence within the simulated reality.

COOPERATIVE SIMULATIONS AND PERFORMANCE ANALYSIS
Due to the demands presented by other simulated games also developed under the LSC and the War Games' Center, the Performance Analysis Method (PAM), which was built for the competitive games, was adapted for cooperative games, although this experience is more recent. After presenting the steps and procedures related to the aforementioned method, we explain how the application of the same tools occurs in cooperative simulation environments. In order to illustrate this, we use the game designed for the Training Course for Security in Major Events (COGEST), of the Police Academy of the State of Rio de Janeiro. Also, we add the case called OLIMPO, in relation to the Brazilian Navy Training for the 2016 Olympic Games.
Throughout the years 2012 and 2014, as a result of the partnership between the Brazilian Naval War College and the State Police Academy of Rio de Janeiro, four editions of simulations were carried out for the Training Course of Operators in Security of Large Events and (COGEST), whose main objective was to improve the interoperability of the country's Security Forces for the 2014 World Cup and the 2016 Olympic Games. Thus, unlike United Nations simulation models, the environment to be simulated had as a characteristic the cooperation among the participants of the game, being thus configured as a multilateral simulation with cooperative characteristics.
When designing the game, the concern of the LSC team was to build a scenario (fictitious and prospective) that would take into account the main situations that could constitute a security threat (both to the public security and national security), during the major events. Thus, during the game, in addition to the exchange of ideas and the testing of the crisis management capacity of the existing institutions, the aim was to stimulate the interinstitutional debate of the security forces (private, public, and defense institutions) and the improvement of the protocols for the crisis management at the tactical and operational levels.
The game's conception anticipated the movement of the actors around three critical moments in which they should organize themselves in order to produce two action plans that should contemplate solutions to the crises presented throughout the game. Before the simulation began, the participants were divided into heterogeneous working groups (WG), containing participants from different hierarchical levels and different organizations. In order to stimulate and facilitate interaction between players, each of the groups counted on the performance of a moderator who had been assigned the control of the debates held in support of the accomplishment of the action plan.
Briefly, the game was proposed through the following phases: PRE-GAME PHASE: Crisis Management Class; Presentation of the case to be simulated; Division of Working Groups (WG). PHASE 1: Elaboration of an Action Plan based on the case presented. PHASE 2: Each Working Group receives a random Action Plan and must elaborate an Implementation Plan for the specific crisis having in mind the possible action plan received. PHASE 3: Cells should, like the use of the Likert scale, classify all elaborated plans. Final Moment: Debriefing-After-Action Report.
In order to obtain reliable results within the Performance Analysis Method, some adaptations were made to the COGEST cooperative model setting. Among the adaptations, the need to assess common vulnerabilities, once the goal was common to the players. Therefore, the decision-making process became an assessment phase where we could consider the actors' effectiveness in resolving the crisis. Thus, analyses of variables, cooperative behavior, dispute settlement and decision making were made.
The steps were divided into two dimensions, the first one linked to the decisionmaking process, in which the analyzed variables reflect cooperative behavior and the mechanisms used to solve the controversy; and the second dimension was the influence and results, to observe the variables that conditioned the final goal delineated throughout the game and therefore, the result.
Representative scheme of the two dimensions within the Performance Analysis for COGEST: For a better understanding of the method and the choice of the variables, it is essential to consider that they are elaborated by the teams or the Working Groups proposed in a period prior to the simulation. Those variables serve as guidelines to be adopted throughout the simulation. From these variables, the experts use the Likert Scale as a method to qualify the pertinence of each one for the game. The evaluation of the players will follow the format of Coherence and Influence in the course of the game, being observed based on the maximum goal and the interaction to reach it. Thus, from the measurement of Coherence and Influence, it is possible to verify the degree of cooperation of the team in the simulation.
In the OLIMPO game, where the Brazilian Navy tested some of the same variables regarding cooperation and compliance with protocols, the same method was applied, as to observe possible gaps between groups and inside them. The variables are listed below: As shown above, principle variables were selected from the control group to be the most important functions of a cooperative framework. Then, sub-variables would give form to the principal ones. Thus, for the variable Acceptability of Ideas, there were subvariables like: despite hierarchy, there was a consensus; decisions are taken collectively and; Ideas are considered from all participants, among others. These results can be seen in the Chart below. In this game, we emphasize that the arguments be observed throughout the simulation.

Cooperative Game
In a cooperative game, the variables/arguments are listed by teams, called Work Groups (WGs) and no longer by an agent chosen by GRUCON. That is, the variables are formulated before the simulation and not during it, predicting the idea of a common goal for a cooperative type game.

Competitive Game
In the second phase of the method, the five arguments will be listed in a software program so that the experts can, using a Likert scale, qualify them for the level of compatibility, in comparison to the actual policies they represent.

Cooperative Game
For a cooperative game, the experts will evaluate, by observing the simulation and the pertinence of the variables for the common objective of the game. This is, therefore, the real scenario for the players and no longer the comparison with the policies as in the competitive game.

Cooperative Game
For a cooperative game, the strategic points have already been outlined by the teams before the simulation. These points are considered as variables and are essential even for the final result. However, at this stage, the experts analyze the consistency of the variables used throughout the game for a common result. That is, the coherence comes from the strategic use delineated by the variables before the simulation and applied to the game.

Cooperative Game
In a cooperative game, phase 4 is also done as a final phase and for the influence analysis. The experts will observe, throughout the achievement of the objectives and the result itself, the team that reached a higher degree of influence in the decision process with the delineation of its strategic variables.

FINAL REMARKS
As presented in the above lines, the Performance Analysis Method was created to produce results for games of a competitive nature, based on the demands of the Brazilian Navy and other governmental agencies. This article used the cases of the Model United Nations and Crisis Game of the Inter-American Defense College to show how we seek a better view of the competitive processes, considering controversies solution a better way to achieve strategic goals. Also, this methodological tool proved to be adaptable and useful to other types of games which are predominantly cooperative, and the goals are synergic, as in the cases explained for the security forces and Brazilian Navy training for the World Cup and the Olympics Games.
In this way, we highlight the functions of the method developed, namely, the analysis of the performance and interaction of the actors in the simulations which can be used in benefit of the strategic planning, as well as the educational perspective observed. Fostering interagency cooperation may be an important result of the adaptation of the competitive model into the cooperative one, as a significant part of the challenges presented nowadays may require actors to generate creative and cooperative solutions. The next steps are connected to the possibility of using game analysis for the creation of short-term scenarios once cooperation and policy feasibility are positively achieved.