# A7: Methodology **Deadline:** Tuesday, November 1, 2022 @ 11:59 pm PT Your assignment this week is comprised of three parts: (1) individually read and review one research paper on warnings or security education (R3), (2) working as a project group, write a methodology section for your course project, and (3) setting up everything you need in preparation for conducting your user study. ### Part 1: Readings Select one of the following papers to read and review: - Bravo-Lillo et al. [Your Attention Please: Designing security-decision UIs to make genuine risks harder to ignore](https://dl.acm.org/doi/pdf/10.1145/2501604.2501610). SOUPS 2013. - Reeder et al. [An Experience Sampling Study of User Reactions to Browser Warnings in the Field](https://dl.acm.org/doi/pdf/10.1145/3173574.3174086). CHI 2018. - Kumaraguru et al. [Getting users to pay attention to anti-phishing education: Evaluation of retention and transfer](https://dl.acm.org/doi/pdf/10.1145/1299015.1299022). eCrime Researchers' Summit 2007. - Sheng et al. [Anti-Phishing Phil: The Design and Evaluation of a Game That Teaches People Not to Fall for Phish](https://dl.acm.org/doi/pdf/10.1145/1280680.1280692). SOUPS 2007. For your selected paper, you should write a short (1-page) review containing the following information: 1. Which paper you read and a 1-2 sentence summary of the paper. 2. A more in-depth outline of the main points of the paper, including the type of study conducted, data collected, analysis methodology used, and conclusions drawn. 3. Any limitations you see of this paper. 4. Possible extensions or follow-up projects relating to this work. 5. Your opinion of the paper, primarily, the quality of the ideas and its potential impact. Prior to reading your selected paper, I strongly recommend checking out [this advice](https://www.eecs.harvard.edu/~michaelm/postscripts/ReadPaper.pdf) on how to read a research paper. **What to submit** A pdf entitled `r3.pdf` containing your review to assignment R3: Security Ed Readings ### Part 2: Methodology Working with your group, write a draft of a methodology section for your final paper. Your methodology section should include all the details about how you plan to conduct your user study, including descriptions of recruitment, conditions (if applicable), screenshots of any tools used, description of your methods, and how you will analyze your data. I'd recommend taking a close look at the methodology sections of the papers in your related work section to see examples of what this can look like. **What to submit** Submit a pdf file named `methods.pdf` that contains your draft methodology section to the assignment A7. ### Part 3: User Study Setup Work with your group to set up everything you will need to run your user study. This might include: putting your survey in Qualtrics, setting up an account and drafting a study on Prolific, implementing or designing tools, scheduling interviews Once you have each individually completed Part 2, work with your group to turn your summaries into a related work section for your final paper. Note that a related should not simply summarize each paper sequentially. Instead, you should connect the papers to describe what is known about the area. You should particularly note how your project differs from or builds upon this prior work. I'd recommend taking a close look at the related work sections of the papers you read to see examples of what this can look like. **What to submit** For any materials on external sites (e.g., Qualtrics or Prolific), you should share them with me through the platform. You should also submit a txt file or pdf named `setup.txt` or `setup.pdf` that describes what you've done. **Note:** next week's assignment will be to conduct your user study. However, you *may not* start collecting any data before your study is formally approved by Pomona's IRB. If your study has already been approved and you want to get a head start on data collection this week (e.g., for an interview-centric project where there isn't a lot of setup, but it will take time to conduct all the interviews), please let me know in advance. ### Part 4: Feedback Answer the following questions: 1. How long did you spend on this assignment? 2. Any comments or feedback? Things you found interesting? Things you found challenging? Things you found boring? **What to submit** A txt file named `feedback.txt` containing your answers to the feedback questions (you may answer these individually or as a group).