School of Computing and Information SystemsThe University of MelbourneCOMP30027, Machine Learning, 2021Project 2: How long will it take to cook this?Task: Build a classifier to predict the cooking time of recipesDue: Stage I: Wednesday 19 May, 5pm UTC+10 (Australian Eastern Standard Time)Stage II: Wednesday 26 May, 5pm UTC+10 (Australian Eastern Standard Time)Stage I: Friday 21 May, 5pm UTC+10 (Australian Eastern Standard Time)Stage II: Friday 28 May, 5pm UTC+10 (Australian Eastern Standard Time)Submission: Stage I: Report (PDF) and code to Canvas; test outputs to Kaggle in-class competitionStage II: Peer reviews and reflection to CanvasMarks: The Project will be marked out of 20, and will contribute 20% of your total mark.Groups: Groups of 1 or 2, with commensurate expectations for each (see Sections 2 and 6).1 OverviewThe goal of this Project is to build and critically analyse supervised Machine Learning methods, to predict thecooking time for recipes based on their steps, ingredients and other features. The cooking time of a recipe hasbeen categorised into three classes, corresponding to quick, medium and slow.This assignment aims to reinforce the largely theoretical lecture concepts surrounding data representation, classifierconstruction, and evaluation, by applying them to an open-ended problem. You will also have an opportunityto practice your general problem-solving skills, written communication skills, and creativity.This project has two stages. The main focus of these stages will be the written report, where you will demonstratethe knowledge that you have gained and the critical analysis you have conducted in a manner that isaccessible to a reasonably informed reader.2 DeliverablesMore details about deliverables are given in the Submission (Section 6).Stage I:1. Report: an anonymous written report, of 1000-1500 words (for a group of one person) or 2000-2500words (for a group of two people)2. Output: the output of your classifiers, comprising predictions of labels for the test instances, submittedto the Kaggle1in-class competition described below.3. Code: one or more programs, written in Python, which implement machine learning models, makepredictions, and evaluate the results.Stage II:1. Peer review: reviews of two reports written by other students, of 200-300 words each (for a group of oneperson) or 300-400 words each (for a group of two people).2. Reflection: a written reflection piece of 400-600 words. This deliverable is individual work.1https://www.kaggle.com/13 Terms of UseThe data has been collected from Food.com (formerly GeniusKitchen), under the provision that any resultingwork should cite this resource:Generating Personalized Recipes from Historical User Preferences. Bodhisattwa Prasad Majumder,Methods in Natural Language Processing and the 9th International Joint Conference on NaturalLanguage Processing (EMNLP-IJCNLP), 2019.This reference must be cited in the bibliography. We reserve the right to mark any submission lacking thisreference with a 0, due to violation of the Terms of Use.Please note that the dataset is a sample of actual data posted to the World Wide Web. As such, it may containinformation that is in poor taste, or that could be considered offensive. We would ask you, as much as possible,to look beyond this to the task at hand. For example, it is generally not necessary to read individual records.The opinions expressed within the data are those of the anonymised authors, and in no way express the officialviews of the University of Melbourne or any of its employees; using the data in an educative capacity does notconstitute endorsement of the content contained therein.If you object to these terms, please contact us (kris.ehinger@unimelb.edu.au orling.luo@unimelb.edu.au) as soon as possible.4 DataThe data files are available via Canvas, and are described in a corresponding README.The recipes are collected from Food.com2, which is a platform that allows the user to publish recipes andcomments on others’ recipes. In our dataset, each recipe contains:• recipe features: name, ingredients, steps, number of steps, and number ofingredients• text features: produced by various text encoding methods for name, ingredients, and steps. Eachfeature is provided as a single file with rows corresponding to the file of recipe features.• class label: the preparation time of a recipe duration (3 possible levels, 1, 2 or 3)You will be provided with training set and a test set. The training set contains the recipe features, text features,and the duration, which is the “class label” of our task. The test set only contains the recipe and text featureswithout the label.The files provided are:• recipe train.csv: recipe features and class label of training instances.• recipe test.csv: recipe features of test instances.• recipe text features *.zip: preprocessed text features for training and test sets, 1 zipped file for each textencoding method. Details about using these text features are provided in README.5 TaskYou are expected to develop Machine Learning models to predict the preparation of a recipe based on its features(e.g. name, ingredients, steps etc.). You will implement and compare different machine learning models andexplore the effective features for this task.2https://www.food.com/2• The training-evaluation phase: The holdout or cross-validation approaches can be applied on the trainingdata provided.• The test phase: the trained classifiers will be evaluated on the unlabelled test data. The predicted labelsof test cases should be submitted as part of the Stage I deliverable.Various machine learning techniques have been (or will be) discussed in this subject (0R, Naive Bayes, DecisionTrees, kNN, SVM, neural network, etc.); many more exist. You may use any machine learning method youconsider suitable for this problem. You are strongly encouraged to make use of machine learning softwareand/or existing libraries (such as sklearn) in your attempts at this project.In addition to different learning algorithms, there are many different ways to encode text for these algorithms.The files in recipe text features *.zip are some possible representations of the name, ingredients and steps ofrecipes we have provided. For example, one of the encoding method is CountVectorizer in sklearn,which converts text documents into “Bag of Words” – the documents are described by word occurrences whileignoring the relative position information of the words. You can use these representations to develop yourclassifiers, but you should also feel free to extract your own features from the raw recipe features, according toyour needs. Just keep in mind that any data representation you use for the text in the training set will need to beable to generalise to the test set.6 SubmissionThe report, code, peer reviews and reflections should be submitted via Canvas; the predictions on test datashould be submitted to Kaggle.Stage I submissions will be open one week before the due date. Stage II submissions will be open as soon asthe reports are available (24 hours following the Stage I submission deadline).6.1 Individual vs. Team ParticipationYou have the option of participating as a “group” of one individual, or in a group of two. In the case that you optto participate individually, you will be required to implement at least 1 and up to 4 distinct Machine Learningmodels. Groups of two will be required to implement at least 3 and up to 5 distinct Machine Learning models,of which one is to be an ensemble model – stacking based on the other models. The report length requirementalso differs, as detailed below:Group size Distinct models required Report length1 1–4 1,000–1,500 words2 3–5 2,000–2,500 wordsIf you wish to form a group of 2, only one of the members needs to register on Canvas by Wednesday 5 May,via the survey “Assignment 2 Group Registration”. For a group of 2, only one of the members needs tosubmit deliverables.Note that once you have signed up for a given group, you will not be allowed to change groups. If you do notregister before the deadline above, we will assume that you will be completing the assignment as an individual,even if you were in a two-person group for Assignment 1.6.2 Stage I: ReportThe report should be 1,000-1,500 words (groups of one person) or 2,000-2,500 words (groups of two people)in length and provide a basic description of:1. the task, and a short summary of some related work32. what you have done, including any learners that you have used, or features that you have engineered.This should be at a conceptual level; a detailed description of the code is not appropriate for the report.The description should be similar to what you would see in a machine learning conference paper.3. evaluation of your classifiers.You should also aim to have a more detailed discussion, which:4. Contextualises the behaviour of the method(s), in terms of the theoretical properties we have identified inthe lectures5. Attempts some error analysis of the method(s)And don’t forget:6. A bibliography, which includes the paper listed in Terms of Use, and other related workNote that we are more interested in seeing evidence that you have thought about the task and investigated thereasons for the relative performance of different methods, rather than in the raw scores of the different methods.This is not to say that you should ignore the relative performance of different runs over the data, but rather thatyou should think beyond simple numbers to the reasons that underlie them, and connect these to the theory thatwe have discussed in this class.Reports must be submitted in the form of a single PDF file. If a report is submitted in any format other thanPDF, we reserve the right to return the report with a mark of 0.To facilitate anonymous peer-review, your name and student ID should not appear anywhere in thereport, including the metadata (filename, etc.).6.3 Stage I: Predictions of test dataTo give you the possibility of evaluating your models on the test set, we will be setting up this project as a Kagglein-class competition. You can submit results on the test set there, and get immediate feedback on your model’sperformance. There is a Leaderboard, that will allow you to see how well you are doing as compared to otherclassmates participating on-line. The Kaggle in-class competition URL and instructions will be announced onCanvas shortly.You will receive marks for submitting (at least) one set of predictions for the unlabelled test dataset into thecompetition; and get basically reasonable accuracy, e.g. better than Zero-R. The focus of this assignment ison the quality of your critical analysis and your report, rather than the performance of your Machine Learningmodels.6.4 Stage II: ReviewsDuring the reviewing process, you will read two submissions by other students. This is to help you contemplatesome other ways of approaching the project, and to ensure that students get some extra feedback. For eachreport, you should aim to write 200-400 words total (200-300 words if you work alone, or 300-400 words ifyou work in a group of two people), responding to three “questions”:• Briefly summarise what the author has done• Indicate what you think that the author has done well, and why• Indicate what you think could have been improved, and whyPlease be courteous and professional in the reviewing process. A brief guideline for reviewers publishedby IEEE can be found https://www.ieee.org/content/dam/ieee-org/ieee/web/org/members/students/reviewer_guidelines_final.pdf.46.5 Stage II: ReflectionsA comprehensive written reflection piece summarising your critical reflection on the following topics within400-600 words.1. the process of completing this project2. things that you are satisfied with and those can be improved in your Stage I deliverables, e.g. modelling,evaluation, analysis and discussion.3. if you have worked in a group of two people, what are the individual contributions?The reflection report is individual and not anonymous. Everyone must submit their own reflection on Canvas.7 Assessment CriteriaThe Project will be marked out of 20, and is worth 20% of your overall mark for the subject. The markbreakdown will be:Report 14 marksPerformance of classifier 2 markReviews 2 marksReflection 2 marksTOTAL 20 marksThe report will be marked according to the rubric, which will be announced via the Canvas.The performance of classifier (2 marks) is for submitting (at least) one set of model predictions to the Kagglecompetition; and get basically reasonable accuracy, e.g. better than Zero-R.Since all of the documents exist on the World Wide Web, it is inconvenient but possible to “cheat” and identifysome of the class labels from the test data using non-Machine Learning methods. If there is any evidence ofthis, the performance of classifier will be ignored, and you will instead receive a mark of 0 for this component.The code will not be directly assessed, but if you do not submit it, it will be assumed that you are attemptingto circumvent the Machine Learning requirement, and you will receive a mark of 0 for the performance ofclassifier.8 Using KaggleThe Kaggle in-class competition URL will be announced on Canvas shortly. To participate the competition:• Each student should create a Kaggle account (unless they have one already) using your Student-ID• You may make up to 8 submissions per day. An example submission file can be found on the Kaggle site.• Submissions will be evaluated by Kaggle for accuracy, against just 30% of the test data, forming thepublic leaderboard.• Prior to competition close, you may select a final submission out of the ones submitted previously – bydefault the submission with highest public leaderboard score is selected by Kaggle.• After competition close, public 30% test scores will be replaced with the private leaderboard 100% testscores.59 Changes/Updates to the Assignment SpecificationsWe will use Canvas to advertise any (hopefully small-scale) changes or clarifications in the assignment speci-fications. Any addendums made to the assignment specifications via Canvas will supersede information containedin this version of the specifications.10 Late SubmissionsLate submissions will bring disruption to the reviewing process. You are strongly encouraged to submit by thedate and time specified above. If circumstances do not permit this, then the marks will be adjusted as follows:• Each day (or part of the day) that the report is submitted after the specified due date and time for Stage I,10% will be deducted from the marks available, up until 7 days (1 week) has passed, after which regularsubmissions will no longer be accepted. A late report submission will mean that your report might notparticipate in the reviewing process, and so you will probably receive less feedback.• Any late submission of the reviews will incur a 50% penalty (i.e. 1 of the 2 marks), and will not beaccepted more than 7 days (1 week) after the Stage II deadline.• Any late submission of the reflection will incur a 50% penalty (i.e. 1 of the 2 marks), and will not beaccepted more than 7 days (1 week) after the Stage II deadline.11 Academic HonestyWhile it is acceptable to discuss the assignment with others in general terms, excessive collaboration withstudents outside of your group is considered cheating. Your submissions will be examined for originality andwill invoke the University’s Academic Misconduct policy here either inappropriate levels of collaboration or plagiarism are deemed to have taken place.6