How I used ChatGPT to replace costly Harvard simulations

Simulations are exciting. They create unprecedented immersion and have a high recall value. But as an educator in an emerging economy, I realize how prohibitive the costs of Harvard simulations can be. An average simulation runs between $15 to $30 per student. For many educational institutions, such expenses are simply unaffordable. Moreover, even when the budget is available, it's often challenging to find a simulation that perfectly aligns with your specific teaching requirements.

However, we are currently witnessing an unprecedented democratization of not only access to educational resources but also the tools to create them.

In a recent class, I decided to experiment with using GPT as a simulation tool for teaching a specific concept. The results were astounding. After sharing my experience on LinkedIn, the post garnered over 50,000 impressions. More than 200 individuals tried the simulation, consuming a total of 141,000 tokens in the process!

Ever since I've been flooded with requests to share how I managed to create this simulation. This article aims to provide a detailed, step-by-step guide on how to create a similar simulation, including the potential challenges and limitations of this approach.

_____________________________________________________________________________________

A request before we start!

Creating such high-quality content is a time-consuming process. If you like it, please consider sharing this with your friends and colleagues. Also, we are soon launching the Precise Words community. If you are interested in joining the community and becoming a contributor, please fill out this very short survey 😎😎

_____________________________________________________________________________________

Now, back to the guide:

Note: This is an updated tutorial, and the Large Language Model (LLM) platform I used to create the simulation has been changed. The initial platform required a certain level of technical knowledge, which made the process daunting for many. This revised tutorial aims to make the process more accessible.

The primary motivation behind creating this simulation was to help students grasp the complexities of using metrics. The central idea was to demonstrate how metrics are always susceptible to manipulation and rigging. Without balancing metrics with social judgment, it becomes nearly impossible to get an accurate picture of what is actually happening on the ground.

Step 1: A simple way to create your own chatbots is Poe which is owned by Quora. You can use different LLMs to power your chatbot. However, for this case, I am relying on GPT-4.

Note: You can run the prompt directly using ChatGPT. However, you need to be mindful of two things. First, the students will be able to see your prompt. Second, and more importantly, students will need to have ChatGPT access.

Step 2: Once you log in to Poe, you will see this screen.

Click on the 'Create Bot' icon on the top left corner.

Step 3: You will see the screen below. Give a name to your bot (It's commissioner007 in this case)

Step 4: Please change the model to GPT-4. While it is possible to run the simulation using the GPT-3.5/ChatGPT model, in my testing the bot behaves rather inconsistently. For example, at times, it'll fail to give options to the player at the start. Other times, it will forget the context after a few iterations.

Step 5: Now here is the most important part. The 'Prompt' is the brain of the simulation. The prompt provides guidance to the language model on how to conduct itself by setting a context and giving it instructions. Your quality of prompts directly impacts how the model behaves.

In this case, this is what my prompt looked like:

______________________________________________________________________________________

Role:

You are GameGPT. You will play a game with MBA students. Note sentences in <> are instructions for you only. Do not tell the user.

Learning Objectives:

The learning objectives of the game are as follows:

Goal Displacement: Understand how metrics can divert effort, making people focus on satisfying those measures at the expense of other crucial organizational goals.

Campbell’s Law: Recognize the corruption pressures and distortions that arise when quantitative social indicators are overused for decision-making.

Game Setup:

The player is the new Police Commissioner of Indrapur in India. You will welcome the player.

Gameplay:

After welcoming, you'll immediately inform the player that the murder of a 17-year-old boy has taken place, and the media and public are up in arms. You'll ask the player to make a decision.

Decision Point: You'll present the player with situations where the player must decide what to do.

Options: Typically, he'll be given four options. Three will involve implementing a new metric to manage employee behavior, and one will be "Something else."

If the player chooses a metric option, provide feedback on the impact.

Only if the player chooses "Something else," the player will be presented with three more metric options and one judgment-based option, such as "Rely on the expertise of senior officers."

Feedback:

After each decision, the immediate consequences are shown to the player. Illustrate how metrics increased attempts at 'gaming the system' by employees to meet their 'target metrics'.

Consequences:

Short-termism: Focusing on short-term metrics can lead to long-term issues.

Increased Bureaucracy: Over-reliance on metrics can lead to more rules and administrative layers.

Organizational Time: Time spent on metrics might exceed their benefits.

Risk-taking: Metrics can discourage initiative and risk-taking.

Cooperation: Metrics can diminish the sense of common purpose and cooperation.

Endgame:

End the game after 4 rounds. Highlight the negative consequences of the player and ask him to reflect upon why they may have happened.

Note: The game has a slight negative bias towards metrics. Players are subtly encouraged to choose metrics over judgment.

______________________________________________________________________________________

This is what the screen would look like with the prompt.

Step 6: Include an 'Intro message' which can help the user get started.

IMPORTANT: Here I need to point out that the prompt structure is extremely important: For example, if the first sentence, “you'll immediately inform the player that the murder of a 17-year-old boy has taken place and the media and public are up in arms”, is not present, the entire conversation will go haywire.

Step 7: Hit save.

You'll automatically be taken to the bot. Just type 'Hello' and the simulation will start.

Challenges that you can expect while using

While this approach to utilizing AI for educational simulations is certainly innovative, it is important to acknowledge that there are challenges and limitations associated with it. Here are some of the issues you are likely to encounter:

Gameplay challenges:

One of the most significant challenges pertains to the game's end conditions. Despite instructing GPT to conclude the game after four rounds, it sometimes continues beyond this point. This can disrupt the flow of the session and lead to confusion among students. It's crucial to find a way to enforce end conditions more effectively through class instructions to ensure a smooth and predictable gameplay experience.

Cost challenges:

The platform used for creating these simulations, Poe, charges a monthly subscription fee of INR 2000. While this isn't a substantial amount, it may pose a challenge for educators, particularly those operating on a tight budget. The issue is compounded by the fact that most educational institutions do not provide reimbursements for these tools as they are not traditionally considered 'academic' resources. In my case, the entire expense was out-of-pocket, which may not be feasible for all educators.

Design Challenges:

Creating an end-to-end simulation using language learning models (LLMs) and prompts is an intricate process that requires considerable time and skill. I personally take to prompting quite naturally. Yet, designing a comprehensive simulation and prototyping it to ensure a successful classroom implementation took a substantial amount of time for me. If you're planning to experiment with these tools, be prepared to dedicate a lot (and I mean a LOT) of your time to the process.

Is there an easy way to learn about more use cases? 

Well, you've already taken the first step by choosing this newsletter. Since, these are emerging use cases, there is no ‘guide’ you can refer to. However, a long-term solution is to create a platform for idea exchange.

The good news is that to facilitate this, I along with some active educators, am soon launching a community where educators to brainstorm, discuss, and create their own use cases and techniques, especially focusing on learning use cases.

If you are interested in joining the community and becoming a contributor, please fill out this very short survey (Not more than 2 minutes. Promise!)😎

If you found this guide helpful, please consider sharing it with your peers and colleagues. Together, we can revolutionize the way we teach and learn, making education more interactive, engaging, and effective.

Till next time!