Assessments Handbook

MGRC20007 Machine Learning for data-driven decision making

2024-25 – UG

Unit Assessment Breakdown

This unit uses two modes of assessment, explained in detail below:

Mode	% of unit  mark	Brief Description	Length
Group Assessment	30%	1,500-word report where students work in groups to solve a business problem using machine learning.   They conduct initial data analysis, develop a preliminary model, and create a comprehensive group report.	1,500 words
Individual Assessment	70%	2,000-word report that builds on the group preliminary work. Each student selects a specific aspect of the project to explore in more depth. This could involve refining the model, addressing limitations, or applying  additional analytical techniques to improve the results.	2,000 words

Assessment 1 – Group Project

Link to Unit Learning Outcomes

This assessment evaluates the following unit intended learning outcomes (ILO):

•     ILO1: Demonstrate an understanding of how decisions are made in

organisations, how sustainable development influences business decision- making, and how business analytics can support decision-making.

•     ILO4: Work effectively in a team to develop data-driven recommendations with the potential for a positive impact in practice.

Overview

In the Group Assignment, students are divided into teams to tackle a data analytics project aimed at predicting customer churn for a bank. This assignment evaluates specific learning outcomes (ILO  1 and  ILO 5) by focusing on how teams apply machine learning techniques to solve a real-world business problem and effectively communicate their findings. Each team member contributes to the collaborative process, from data exploration to model development, ensuring a comprehensive understanding of the project and its implications. Collaboratively, they conceptualise and implement the project, applying data mining techniques effectively.

The deliverables for this assignment include:

•     Report (1,500 words): Each team crafts a report similar to a management consultancy   document.   It   outlines   the   project   strategy,   encountered challenges, team reflections, and the outcomes achieved.

Submission

The group report will be submitted on Turnitin, and further details will be provided closer to the time. The due date for the group report and presentation will be 31st October 2024 13:00 GMT.

Assessment Weight

30% of the total unit mark

Assessment 2 – Individual Coursework

Link to Unit Learning Outcomes

This assessment covers the following unit learning objectives:

•    ILO2:  Use  mathematical  tools  to  formulate  decision-making   problems, develop solutions, and provide recommendations based on analytics.

•    ILO3: Design and develop suitable predictive analytics solutions to business decision-making problems within the limits of time and resources available.

•    ILO4: Implement and evaluate a variety of predictive models and improve on their design to meet business needs and requirements.

Assessment Instructions

The purpose of this assessment is to assess your ability to independently apply and extend the machine learning concepts covered in this unit, specifically addressing ILOs 1 and 5. You are required to submit a 2,000-word report that builds on the group  project  by  refining  and  enhancing  the  customer  churn  prediction  model developed in the group work. Your report should demonstrate a deep understanding of the techniques used to improve the model, discuss the ethical considerations involved, and analyse how the refined model can be strategically applied within a real-world banking context. This assignment challenges you to critically evaluate the   model   effectiveness/accuracy,    address   potential    biases,   and   propose actionable business insights based on your findings. We encourage the integration of theoretical knowledge with practical application, reflecting real-world business analytics challenges in an individual context.

Assessment Weight

70% of the total unit mark

Important information about the report

The  report  should  be 2,000  words  (+/-  10%) in  length.  Please  keep  in  mind the following points:

•   This is not an essay assignment. You should not describe or analyse theory and models in isolation. You will be assessed primarily on how you apply the concepts and models from this unit to a real analytics problem, evaluate its effectiveness, identify and recommend improvements that could be made.

•    However, you should include academic references where relevant e.g., the source of models used in your report, supporting data. Please note that you cannot cite the lecture slides! Any quotations from such sources should be properly referenced including page numbers using Harvard referencing style, with full details included in the references section.

•    It is recommended that your work should have between 10 to 20 references. Please use Harvard referencing throughout.

•   Throughout the report, you will need to focus on both the presentation and clarity of any models, diagrams and tables, as well as the quality of your analysis. Consider  how  to  best  present  your  work  to  make  it  look  as professional as possible e.g., using a contents page.

•   Tables and Figures must be labelled with a caption, “Figure 1: Diagram of … .” etc. with the text referring to the Figure, or table, as Figure 1, etc.

•    Reading beyond the course materials is vital.

•   The  use  of  graphs  and  diagrams  to  illustrate  your  analysis  is  strongly encouraged.

•   You are reminded that this assignment is about Business Analytics, not data science nor statistics. The mathematical justification and specific number of analyses you applied is not as important as your ability to conduct a clear analysis of the business implications of those data analytics technique.

Report Structure

Please  note  that  the  structure  below  is  the  recommended  format  for  the assessment.

Recommended Report Structure for the Individual Role-based Assignment

1.  Title Page

•    Title of the report

•     Student ID (not your name)

•     Course title

•     Date of submission

•    Word count

2.  Executive summary

•    A brief summary of the report purpose, key findings, and conclusions.

3.  Introduction

•     Overview  of  the  group   project   and  the   machine   learning   model developed. Which model did your group use and why?

•     Objectives of the report.

•     Brief introduction to the importance of customer churn prediction in the banking sector. Use references to support your arguments.

4.  Model improvement

•     Detailed description of the enhancements  made to the original model (e.g., hyperparameter tuning, feature engineering, cross-validation).

•    Justification for the chosen methods and techniques.

•     Discussion of the impact of these improvements on model performance, including any new metrics or validation results.

5.  Ethical considerations

•    Analysis of potential biases in the model (e.g., gender, age, geography) and their implications.

•     Discussion on the ethical use of predictive models in banking, particularly in customer retention.

•     Proposed  strategies  to  mitigate  identified  biases  and  ensure  ethical application of the model.

6.  Strategic application

•    Application of the refined model to a specific business scenario within the bank (e.g., targeting high-risk customers for retention or SMEs).

•    Analysis of how the model predictions can inform. strategic decisions.

•     Discussion of the potential business impact of these decisions, supported by data from the model.

7.  Critical reflection

•     Personal  reflection on the learning  process and the challenges faced during the project.

•     Evaluation of the strengths and weaknesses of the model and the overall approach.

•     Suggestions for further improvements or alternative approaches.

8.  Conclusion

•     Summary of key findings and insights from the report.

•     Reiteration of the importance of model refinement, ethical considerations, and strategic application in business analytics.

9.  References

10.Appendices (if applicable)

•    Additional data, charts,  Python scripts, or other material referenced in the report but not included in the main body.

You  should  also  make  sure  that  you  are  fully  aware  of  the School's policy  on plagiarism. You should be aware that you cannot later claim that you did not know the rules and regulations. Copying material from similar essays that can be found on essay websites  is  not  acceptable  and  can  lead  to  disciplinary  action.  See

https://www.bristol.ac.uk/students/support/academic-advice/academic- integrity/plagiarism/  for full information.

Various websites claim that they help students by showing them what is expected on a typical assignment such as in Business Analytics. These tacitly encourage plagiarism and copying, which does not demonstrate true understanding. It is more important  to  develop  your  own  voice  and  your  own  abilities in  writing   and research, and to show that you can see how operations function in any real-world setting. All assignments are scanned via plagiarism detection software

At the University of Bristol, while recognising the potential of AI technologies in enhancing  learning,  the  emphasis  is  on  ethical  usage  and  academic  integrity. Students are expected to use AI tools, like ChatGPT, responsibly, adhering to the university's  guidelines  that  discourage  dependency  on  such  technologies  for completing academic tasks. For detailed information on the correct use of AI in your studies, including the university's policies and recommendations, please refer to the Using AI at University guideprovided by the Study Skills team.

Submission

Each student must submit their completed portfolio assignment electronically via Canvas. Assessment due date is 28th  November 2024 at 13:00 (UK time).