Experiment 22 - Monte Carlo Simulation

Department of Electrical Engineering & Electronics

September 2023, Ver. 3.6

Experiment specifications

Module(s)	ELEC224 / ELEC273
Experiment code	22
Semester	1
Level	2
Lab location	PC labs, third floor/fourth floor, check the timetable
Work	Individual
Timetabled time	7 hrs
Subject(s) of relevance	Probability and Statistics
Assessment method	One lab report per person. Please use the available template in CANVAS for Experiment 22 assignment.
Submission deadline	At 23:59, 7 days after the date of the laboratory, submitted in Microsoft Word or PDF format via CANVAS only.

Important: Marking of all coursework is anonymous. Do not include your name, student ID number, email or any other personal information in your report or in the name of the file submitted via CANVAS. A penalty will be applied to submissions that do not meet this requirement.

Instructions:

● Read this script carefully before attempting the experiment.

● Review MATLAB before attempting the experiment and use it for all the re- quired coding and graphs. Check CANVAS for MATLAB resources (Learning Resources→ Supporting Material folder).  See online material and resources as well.

● Keep a record of all code, graphs and results.

● The code must be well-structured and organised (to get a better mark).  Use the concept of functions for better code re-usability and managment.

● The code must be provided for every requirement with appropriate explana- tion. Use MATLAB commenting  (%) to emphasise on and explain relevant

code lines.

● For every requirement, the code must run and screenshots of the results must be provided.

● Any change to the code needs to be reinserted every time.

● Be sure to reference any resource you have used in writing your report.

● Use your time wisely. Finish as many tasks as you can during the lab (with demonstrators support). If you cannot finish all tasks today, you can complete

your work at home later on.

● If you have any feedback on your laboratory experience today, please write it down on the last page of the script.

1 Learning outcomes

The purpose of this experiment is to develop,  explore and test Monte  Carlo techniques in simulating and finding solutions to real-life random processes.  MATLAB will be used as the tool to do the tests of the experiment,  but it is not the main learning outcome  (i.e.   the experiment is not about MATLAB).

2 Introduction

The Monte Carlo method is a numerical method of solving mathematical problems by the sim- ulation of random variables.  The name Monte  Carlo was applied to a class of mathematical methods first by scientists working on the development of nuclear weapons in Los Alamos in the 1940s. The essence of the method is the invention of games of chance whose behaviour and outcome can be used to study some interesting phenomena. While there is no essential link to computers, the effectiveness of numerical or simulated gambling as a serious scientific pursuit is enormously enhanced by the availability of modern digital computers [1].

The term “Monte Carlo” refers to procedures in which quantities of interest are approximated by generating many random realisations of a stochastic process and averaging them in some way. In statistics, the quantities of interest are the distributions of estimators and test statis- tics, the size of a test statistic under the null hypothesis, or the power of a test statistic under some specified alternative hypothesis [2].

How can we use Monte Carlo techniques to find the sampling distribution of an estimator?  In the real world, we usually observe just one sample of a certain size N, which will give us just one estimate. The Monte Carlo experiment is a lab situation, where we replicate the real world study many (R) times.  Every time, we draw a different sample of size N from the original population.  Thus, we can calculate the estimate many times and any estimate will be a bit different.   The empirical distribution of these many estimates approximates the true of the estimator. A Monte Carlo experiment involves the following steps [3]:

(1) Draw a (pseudo) random sample of size N for the stochastic elements of the stochastic model from their respective probability distribution functions.

(2) Assume values for the parts of the model or draw them from their respective distribution function.

(3) Calculate the parts of the statistical model.

(4) Calculate the value (e.g. the estimate) you are interested in.

(5) Replicate step (1) to (4) R times.

(6) Examine the empirical distribution of the R values.

The Monte Carlo approach is relevant to different scientific disciplines and problems including (but not limited to) the following areas [4]:

● Physical sciences: computational physics, physical chemistry, quantum chromodynam- ics, statistical physics, molecular modelling, particle physics and galaxy modelling.

● Designs and visuals/Computer graphics: global illumination, photorealistic images of virtual 3D models, video games architecture and design, computer generated films and special effects in cinema.

● Finance and business/Operations research: evaluating investments in projects at a business unit, evaluating financial derivatives, construction of stochastic or probabilistic financial models and in enhancing the treatment of uncertainty in the calculation.

● Telecommunications: planning a wireless network, generating user patterns and their states, testing the probability of losing information in a network whether it is below a certain threshold.

● Games: game playing related artificial intelligence theory.

3    The Practical Work

Penalty kicks are a critical time of decision-making for both the goalkeeper and the penalty taker in football matches.  Given that, for most professional games, the average number of goals scored is around 2.5, a penalty kick can have a major influence on the outcome of a match. Penalty kicks may reach speeds near 125 mph and is usually over within a quarter of a second. Thus, the goalkeeper must make a decision on how to stop the shot before the ball is struck. Statistics show that goalkeepers will most often jump to the left or right, hoping to guess cor- rectly the position to block the kick [5,6].

Consider the situation of a football goal and a blindfolded person trying to shoot the ball from the penalty spot and score a goal.  Let us assume that the goal has dimensions L and W as shown in Figure 2, and there is an imaginary circle that circumscribes the goal.  Two cases will be considered: first when there is no goalkeeper and second when there is a goalkeeper saving the ball.

Figure 2: The goal arrangement.

3.1 Part I: No Goalkeeper Tests (45 Marks)

In this case, there is no goalkeeper, and it is just the penalty taker against the goal. You need to model each shot by treating the co-ordinates of the ball in the goal plane as random variables (i.e. ignore the trajectory of the ball).

● Task-1. If a large number of shots is attempted, derive a numerical value for the fraction of balls entering the goal to the total number of balls in the circular area. Assume the penalty taker is blindfolded (i.e. the shots are uniformly distributed within the circle). [5 Marks]

● Task-2. Design and write a computer programme to find the probability of scoring by simulating N random penalty shots and repeating this experiment R times and taking the mean of the attempts.  Let N and R be inputs to your code.  Use a uniform random number generator in the simulation. [8 Marks]

● Task-3. Produce an appropriate scatter plot illustrating your experiment for N = 1,000 and R = 1, using red crosses to indicate score (i.e.  balls on target) and blue circles to indicate miss (i.e. balls off target). Insert an appropriate legend. [5 Marks]

● Task-4. For R = 5, find the probability of scoring for N = 100, N = 1,000, N = 10,000 and N = 100,000. Plot the probability against the value of N. Comment on the shape of the plot, making reference to the theoretical probability calculated in Task-1. Remember to label the axes and to insert an appropriate caption in your report. [7 Marks]

● Task-5. For N =  1,000, find the probability of scoring for R = 5 times, R = 10 times, R = 15 times and R = 20 times. Plot this probability against the value of R.  Comment on the shape of the plot. [5 Marks]

● Task-6. Compare with appropriate explanation the two cases of Task-4 and Task-5 based on the obtained probability plots. [5 Marks]

● Task-7. Repeat Task-2 to Task-6 using a normal (Gaussian) random number gener- ator. Assume the distribution to be centred at the centre of the circle and with standard deviation equal to the radius.  Comment (with appropriate explanation) on the differences between the results of the two cases. [10 Marks]

3.2 Part II: With Goalkeeper Tests (25 Marks)

Consider now the above case but with a goalkeeper.  The goalkeeper can assume one of five possible actions (see Figure 3):  stays in the middle, jumps to the upper left corner, jumps to the upper right corner, jumps to the lower left corner or jumps to the lower right corner.  A ball is saved if the goalkeeper guesses the correct ball position.  The goal area can be divided into five corresponding regions as shown in the figure.

Figure 3: Five possibilities of a goalkeeper action to a penalty shoot-out.

● Task-8. Assuming that the goalkeeper action is modelled as a uniform. random process, what is the probability of scoring a goal if the penalty taker kicks 100 balls with uniform. random distribution within the circle, as before. Increase the kicks to 1000. Compare the probability values with the case where no goalkeeper was in the goal (Task-1 and Task-3 above). [25 Marks]

Note: For Part II, you need to provide code, plots, explanations & comments as in Part I.

4 Review Questions (30 Marks)

Q1. In terms of what you have done in this experiment,  comment on the  advantages and disadvantages (or drawbacks) of the Monte Carlo experiment. [5 Marks]

Q2. Discuss the ways in which the  above model could be made more  accurate  and realis- tic. [10 Marks]

Q3. With reference to Task-7, discuss the effect of changing the standard deviation of the Gaussian distribution on both the accuracy and precision of the penalty shots. [5 Marks]

Q4. If a large number of balls are kicked on the goal (i.e.  if N is sufficiently large), the value of π can be estimated using (some function of) the ratio of the number of scores to the total number of the shots.  Hence, find the relation that estimates the value of π .  Verify this using your results for both uniform. and Gaussian distributions. [10 Marks]

5    Report Writing and Marking Scheme

This experiment is assessed by means of a report.  Reports that get 70% and above are first- class reports only. Please refer to Appendix A to read about report marking descriptors.

The marking scheme for the report of this experiment is as follows:

● Results of Part I with code, plots, explanation and comments: 45 Marks

● Results of Part II with code, plots, explanation and comments: 25 Marks

● Discussions and Conclusions section (including review questions): 30 Marks

6 Plagiarism and Collusion

Plagiarism and collusion or fabrication of data is always treated seriously, and action appro- priate to the circumstances is always taken.   The  procedure  followed  by  the  University  in all cases where plagiarism, collusion or fabrication is suspected is detailed in the University’s Policy for Dealing with Plagiarism, Collusion and Fabrication of Data, Code of Practice on Assessment, Category C, available on https://www.liverpool.ac.uk/media/livacuk/tqsd/ code-of-practice-on-assessment/appendix_L_cop_assess.pdf.

Follow the following guidelines to avoid any problems:

(1) Do your work yourself.

(2) Acknowledge all your sources.

(3) Present your results as they are.

(4) Restrict access to your work.

Facts about penalty shoot-outs:

● Over 10 recent world cups’ penalty shoot-outs, 80% were scored successfully [5].

● A study for 1,000 penalty shoot-outs has shown that 74.7% of the kicks were successful, 18.2% were saved by the goalkeeper, 3.5% missed the goal and 3.6% hit the woodwork and ended with no goal [6].

● The most successful football team in penalty shoot-outs is Germany.  They lost only one shoot-out throughout their history in recorded matches [7].

● England footballteam has bad penalty shoot-out record in major international matches [7].

References

[1] G Rubino and B Tuffin,  “Rare  Event  Simulation using Monte Carlo Methods”, Wiley, 2009.

[2] C Lemieux, “Monte Carlo and Quasi-Monte Carlo Sampling”, Springer, 2009.

[3] M Kalos and P Whitlock, “Monte Carlo Methods”, Wiley, 2004.

[4] G Fishman, “Monte Carlo Concepts, Algorithms and Applications”, Springer, 1996.

[5] M Bar-Eli and O Azar, “Penalty kicks in soccer: an empirical analysis of shooting strategies and goalkeepers preferences”, Soccer & Society, 2009.

[6] Just      About      Football,       “Penalty      Kick      Statistics       and      Success      rates”, http://justaboutfootball.blogspot.com/2009/02/penalty-kick-statistics.html, 2009.

[7] J Billsberry, “Shootouts Alternatives”, http://www.penaltyshootouts.co.uk/alternatives.html, 2010.