Problem Set 2

1.    For the following examples, what do you think the level of randomization should be?

Hint: When answering this question, first think about whether individual level randomization is possible. To determine that, you need to think through any potential concerns about logistics, fairness, balance, spillovers as well as the level of the outcome of interest. If individual level randomization is not possible, what are other potential levels you can use and why?

a.    In a field study, in a  large amusement park, participants rode a roller coaster, were photographed during the ride, and later chose whether to purchase a print of the photo. The researcher wants to measure the effect of pricing strategy (a fixed price of $10 vs pay want you want) on profits.

b.   Uber wants to measure the effect of increasing driver’spay (paying drivers 10% more per ride vs. paying current amounts) on driver’s hours worked.

c.    Oberweiss, an ice-cream cream chain, wants to know the effect of offering free samples on ice-cream sales.

2.   A  researcher plans to ask six subjects to donate either 30 or 60 minutes of their  time. Researcher considers the following three methods for randomization:

*Coin flip : ask 30 minutes of time if coin comes up heads, ask 60 minutes otherwise

*Ordered : ask the 3 subjects who arrived at the lab first for 60 minutes of their time and the last 3 subjects for 30 minutes of their time

*Lottery : write 30 on 3 slips of paper and 60 on another 3 slips of paper; put the slips into envelope,sealand shuffle the envelopes, randomly pick one envelope (without replacement) before talking to a subject and ask the number written on the slip.

a. Discuss the advantages and/or disadvantages of these methods. Which of these methods do you recommend to the researcher?

b. What is the distribution of the average assigned number of minutes if the coin flip method is used?  What is the distribution of the average assigned number of minutes if the lottery method is used?

c.  How will your answer to part (a) change if there are 600 subjects instead of 6 subjects?

3.   Suppose you are interested in understanding how 7-hour workdays compared to 8-hour workdays affect the productivity of your workers using an experiment. That is, you will randomly assign 100 workers to have 7-hour workdays (treatment) and 100 workers to have 8-hour workdays (control). You have data on your employees’ gender, age, years of experience, and current productivity.

a.   Would you choose to do a complete random assignment or a stratified random assignment? Explain your reasoning.

b.   If you can only stratify on one variable, which variable would you choose to stratify on? Why? Do you  need to do any  modifications on this variable before doing the stratification?

c.    If you can choose more than one variable to stratify on, which additional variable(s) would you choose to stratify on? Why? Do you need to do any modifications on these variable(s) before doing the stratification? Are there any concerns about choosing many variables to stratify on?

d.   Would  your  answer  to  part  (a)  be  different if  you  were randomly assigning 1000 workers  to  have  7-hour  workdays  (treatment)  and 1000 workers  to  have  8-hour workdays (control)? If yes, why/how?

4.   Your research team is interested in looking at the impact of providing students a tutor. These tutors work with children in grades 2, 3 and 4 who are identified as falling behind their peers. Through  a  pilot  survey,  we  know that  the  average  test  scores  of  these  students  before receiving tutoring is 26 out of 100, with a standard deviation of 20.

Note:  For  this  question,  please   use  the  formulas  that  we   learned  in  class  and  do  the calculations yourself instead of using the software. You are allowed to use excel or online calculators to calculate the relevant values of the student’st-distribution. Show all your work. It is ok if your answers are not round numbers.

a.    If you have 1000 students and you randomly assign half of them to the control group and half of them to the treatment group, what is the minimum detectable effect you can measure when the level of significance is 0.05 and power you want is 80%?

b.   If you have 2000 students and you randomly assign half of them to the control group and half of them to the treatment group, what is the minimum detectable effect you can measure when the level of significance is 0.05 and power you want is 80%?

c.    Based on  parts a and  b, will you need larger or smaller samples to measure smaller effect sizes?

d.   If you have 1000 students and you randomly assign 3/4th of them to the control group and 1/4th of them to the treatment group, what is the minimum detectable effect you can measure when the level of significance is 0.05 and power you want is 80%?

e.    Based on your answer in parts a and d, will you prefer unequal assignment allocations or equal

assignment allocations? When might a researcher prefer unequal assignment allocations?

f.    Suppose you control for some  student characteristics  which reduces the  residual standard deviation to 10. If you have 1000 students and you randomly assign half of them to the control group and half of them to the treatment group, what is the minimum detectable effect you can measure when the level of significance is 0.05 and power you want is 80%?

g.    Based on your answer in parts a and f, will you be able to detect smaller effect sizes when the standard deviation is lower or higher?

h.   Your research team decides to randomize at the school level instead of at the individual level. So, you are doing a clustered randomization. Suppose you have 20 schools in total and each school has 200 students. You randomly assign half of the schools to control and half of them to treatment. Continue to assume a standard deviation of 20.

i.     What  is  the  minimum  detectable  effect  you  can  measure  when  the  intracluster correlation is 0.05 and the level of significance is 0.05 and power you want is 80%?

ii.   What  is  the  minimum  detectable  effect  you  can  measure  when  the  intracluster correlation is 0.20 and the level of significance is 0.05 and power you want is 80%?

iii.  What  is  the  minimum  detectable  effect  you  can  measure  when  the  intracluster correlation is 0.50 and the level of significance is 0.05 and power you want is 80%?

iv.   Based  on  previous  parts, will you  be  able to detect smaller effect sizes when the intracluster correlation is lower or higher?