**CS 190 Senior Seminar** Edmunds 129 Sections 1: Tuesday 11:00 AM to 12:15 PM Sections 2: Thursday 11:00 AM to 12:15 PM Instructor - Anthony J. Clark - Edmunds 127 - Office hours: + Wed from 9 to 11 + Fri from 9 to 11 *Reading, discussion and presentation of research papers in an area of computer science. Each student will write a survey paper and must attend the Computer Science Colloquium as requested in CS major requirement.* # Goals and Grading You are expected to complete the following requirements: 1. Complete the $\LaTeX$ exercise. 2. Participate in weekly in-class discussions and complete questionnaires. 3. Work with me to create a presentation for two technical articles. 4. Present two technical articles. 5. Complete all writing milestone listed on the calendar. Each week will involve - presenters (1 or 2 presenters per week) + reading the article(s) at least three days before presenting, + meeting with me and discussing presentation plans at least two days before presenting, + sending me your slides at least one day before presenting, + giving a 20 to 25 minute presentation, and + ending presentations with three discussion questions; - audience members + reading the article(s) at least two days before the presentation, + (optionally) adding comments using Hypothesis, + completing a questionnaire one day before the presentation, and + participating in our group discussion. | | Requirements | | - | --------------------------------------- | | A | Pass all five requirements listed above | | B | Pass final paper and both presentations | | C | Pass final paper and one presentation | | D | Pass final paper | | F | *Otherwise* | [Grade table] You will have specific rubrics (on [gradescope](https://www.gradescope.com/)) for what constitutes a *"pass"* for each requirement. # Technical Article Options I've grouped articles into categories below. I may not have found the best fit for each article, so please look at more than just the title when making selections. You must present papers from two different categories. ## Data Ethics - [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation (arXiv)](https://arxiv.org/abs/2205.00501) - [Towards fairer datasets (ACM)](https://dl.acm.org/doi/abs/10.1145/3351095.3375709?casa_token=tpzTDK23Om8AAAAA:8T3Wzu_3GxwZr0sK2r_giFuwh8RBifa3NGSl4NkTdinFj9muTVcrOe34sP6-tkDBZ7I1OV5vhB07fUg) - [Producing wrong data without doing anything obviously wrong! (ACM)](https://dl.acm.org/doi/10.1145/2528521.1508275) - [Never Mind the Trolley: The Ethics of Autonomous Vehicles in Mundane Situations (Springer)](https://link.springer.com/article/10.1007/s10677-018-9896-4) ## Governance - C++ (use all four links) + [Standard C++ Foundation](https://isocpp.org/about) + [How To Submit a Proposal : Standard C++](https://isocpp.org/std/submit-a-proposal) + [List of proposals](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/) + [P2000R3 Direction for ISO C++ (PDF)](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2000r3.pdf) - Rust (use all four links) + [Proposal - The Rust Language Design Team](https://lang-team.rust-lang.org/initiatives/process/stages/proposal.html) + [Rust Foundation](https://foundation.rust-lang.org/) + [Rust proposals](https://rust-lang.github.io/rfcs/) + Find a specific proposal (I'll help) - Python (use all three links) + [PEP 0 – Index of Python Enhancement Proposals](https://peps.python.org/) + [PEP 13 – Python Language Governance](https://peps.python.org/pep-0013/) + Find a specific proposal (I'll help) - GNU (use all four links) + [Philosophy of the GNU Project](https://www.gnu.org/philosophy/philosophy.html) + [The GNU Assembly — Governance](https://gnu.tools/en/documents/governance/) + [Rethinking the governance of the GNU Project](https://lwn.net/Articles/802985/) + [The Structure and Administration of the GNU Project](https://www.gnu.org/gnu/gnu-structure.en.html) + Feel free to dive into any additional [Essays and Articles](https://www.gnu.org/philosophy/essays-and-articles.html) ## Software Engineering - [Exploring the intersection between software industry and Software Engineering education - A systematic mapping of Software Engineering Trends (ScienceDirect)](https://www.sciencedirect.com/science/article/pii/S0164121220301667) - [Software engineering in start-up companies: An analysis of 88 experience reports (Springer)](https://link.springer.com/article/10.1007/s10664-018-9620-y) - Software Engineering Education (both papers) + [The Impact of Combining Storytelling with Lecture on Female Students in Software Engineering Education (IEEE)](https://ieeexplore.ieee.org/abstract/document/9453992) + [Software Engineering Education: Challenges and Perspectives (IEEE)](https://ieeexplore.ieee.org/abstract/document/9125353) - [Software Engineering for Machine Learning: A Case Study (IEEE)](https://ieeexplore.ieee.org/abstract/document/8804457) - Diversity in Software Engineering + [The Diversity Crisis in Software Development (IEEE)](https://ieeexplore.ieee.org/abstract/document/9354402) + Choose any other article from the same [journal issue](https://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=9354388&punumber=52) - [Impact of design patterns on software quality: a systematic literature review (Wiley)](https://ietresearch.onlinelibrary.wiley.com/doi/full/10.1049/iet-sen.2018.5446) - [Internet of Things (IoT) for Next-Generation Smart Systems: A Review of Current Challenges, Future Trends and Prospects for Emerging 5G-IoT Scenarios (IEEE)](https://ieeexplore.ieee.org/abstract/document/8972389) # Schedule The schedule combines sections 1 and 2. Section 1 students need only look at Tuesdays, and section 2 students need only look at Thursdays. Tuesday Aug 30, 2022: Prof. Clark - Section 1 - Introduction and logistics - Article selection - How to find, read, and write - [You should be reading academic computer science papers - Stack Overflow Blog](https://stackoverflow.blog/2022/04/07/you-should-be-reading-academic-computer-science-papers/) - Read section How to Read Articles and section Presentation Advice and section Presentation Requirements Thursday Sep 1, 2022: Prof. Clark - Section 2 - Introduction and logistics - Article selection - How to find, read, and write - [You should be reading academic computer science papers - Stack Overflow Blog](https://stackoverflow.blog/2022/04/07/you-should-be-reading-academic-computer-science-papers/) - Read section How to Read Articles and section Presentation Advice and section Presentation Requirements Tuesday Sep 6, 2022: David and Emily - Section 1 - [Towards fairer datasets (ACM)](https://dl.acm.org/doi/abs/10.1145/3351095.3375709?casa_token=tpzTDK23Om8AAAAA:8T3Wzu_3GxwZr0sK2r_giFuwh8RBifa3NGSl4NkTdinFj9muTVcrOe34sP6-tkDBZ7I1OV5vhB07fUg) Thursday Sep 8, 2022: Carl and Irmak - Section 2 - [Towards fairer datasets (ACM)](https://dl.acm.org/doi/abs/10.1145/3351095.3375709?casa_token=tpzTDK23Om8AAAAA:8T3Wzu_3GxwZr0sK2r_giFuwh8RBifa3NGSl4NkTdinFj9muTVcrOe34sP6-tkDBZ7I1OV5vhB07fUg) Tuesday Sep 13, 2022: Guy and Ino - Section 1 - [Never Mind the Trolley: The Ethics of Autonomous Vehicles in Mundane Situations (Springer)](https://link.springer.com/article/10.1007/s10677-018-9896-4) Thursday Sep 15, 2022: Jiahao and Millie - Section 2 - [Never Mind the Trolley: The Ethics of Autonomous Vehicles in Mundane Situations (Springer)](https://link.springer.com/article/10.1007/s10677-018-9896-4) Friday Sep 16, 2022: Advisor Selection - Senior thesis and project students must find an advisor Sunday Sep 18, 2022: $\LaTeX$ - Read section General Writing Advice - $\LaTeX$ assignment due on [gradescope](https://www.gradescope.com/) Tuesday Sep 20, 2022: Hadi and Jason - Section 1 - [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation (arXiv)](https://arxiv.org/abs/2205.00501) Thursday Sep 22, 2022: Jia and Nathan Dailey - Section 2 - [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation (arXiv)](https://arxiv.org/abs/2205.00501) Tuesday Sep 27, 2022: Claire and Hannah - Section 1 - [Producing wrong data without doing anything obviously wrong! (ACM)](https://dl.acm.org/doi/10.1145/2528521.1508275) Thursday Sep 29, 2022: Alan and Nathan Getachew - Section 2 - [Producing wrong data without doing anything obviously wrong! (ACM)](https://dl.acm.org/doi/10.1145/2528521.1508275) Sunday Oct 2, 2022: Topic and References - Topic selection and reference list due on [gradescope](https://www.gradescope.com/) - Find eight peer-reviewed articles related to your topic - Read section How to Find Articles Tuesday Oct 4, 2022: Emily and Ino - Section 1 - [The Diversity Crisis in Software Development (IEEE)](https://ieeexplore.ieee.org/abstract/document/9354402) - And an addition paper from the same [journal issue](https://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=9354388&punumber=52) chosen by Emily and Ino (will appear here) + [Gendered Experiences of Software Engineers During the COVID-19 Crisis](https://ieeexplore.ieee.org/document/9268454) Thursday Oct 6, 2022: Carson and Jia - Section 2 - [Impact of design patterns on software quality: a systematic literature review (Wiley)](https://ietresearch.onlinelibrary.wiley.com/doi/full/10.1049/iet-sen.2018.5446) Tuesday Oct 11, 2022: David and Jason - Section 1 - [Impact of design patterns on software quality: a systematic literature review (Wiley)](https://ietresearch.onlinelibrary.wiley.com/doi/full/10.1049/iet-sen.2018.5446) Thursday Oct 13, 2022: Alan and Nathan Daily - Section 2 - [The Impact of Combining Storytelling with Lecture on Female Students in Software Engineering Education (IEEE)](https://ieeexplore.ieee.org/abstract/document/9453992) - [Software Engineering Education: Challenges and Perspectives (IEEE)](https://ieeexplore.ieee.org/abstract/document/9125353) Sunday Oct 16, 2022: Bibliography - Annotated bibliography due on [gradescope](https://www.gradescope.com/) - Summarize eight articles in one to three sentences each - Read section General Writing Advice - Read [How to Prepare an Annotated Bibliography](https://guides.library.cornell.edu/annotatedbibliography) (Tuesday Oct 18, 2022): Fall break Thursday Oct 20, 2022: Carl and Carson - Section 2 - [P2000R3 Direction for ISO C++ (PDF)](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2000r3.pdf) Tuesday Oct 25, 2022: Claire and Melat; Nick - Section 1 - The following for Claire's and Melat's presentation + [GNU Manifesto](https://www.gnu.org/gnu/manifesto.html) + [Why open source misses the point of free software](https://www.gnu.org/philosophy/open-source-misses-the-point.html) - [P2000R3 Direction for ISO C++ (PDF)](https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2000r3.pdf) for Nick's presentation Thursday Oct 27, 2022: Millie and Sam - Section 2 - Python PEP TBD by presenters (will appear here) Sunday Oct 30, 2022: Outline and Intro - Full draft outline due on [gradescope](https://www.gradescope.com/) - Full draft of introduction section - Read section How to Write an Introduction Tuesday Nov 1, 2022: Peer-Review - Section 1 - Read section Peer-Review Process - Completed peer-review due on [gradescope](https://www.gradescope.com/) Thursday Nov 3, 2022: Peer-Review - Section 2 - Read section Peer-Review Process - Completed peer-review due on [gradescope](https://www.gradescope.com/) Tuesday Nov 8, 2022: Hadi and Jan - Section 1 - Python PEP TBD by presenters (will appear here) Thursday Nov 10, 2022: Nathan Getachew and Jiahao - Section 2 - [Internet of Things (IoT) for Next-Generation Smart Systems: A Review of Current Challenges, Future Trends and Prospects for Emerging 5G-IoT Scenarios (IEEE)](https://ieeexplore.ieee.org/abstract/document/8972389) Tuesday Nov 15, 2022: Hannah and Melat - Section 1 - [Internet of Things (IoT) for Next-Generation Smart Systems: A Review of Current Challenges, Future Trends and Prospects for Emerging 5G-IoT Scenarios (IEEE)](https://ieeexplore.ieee.org/abstract/document/8972389) Wednesday Nov 16, 2022: First Draft - Completed first draft due on [gradescope](https://www.gradescope.com/) - Read section Survey Paper Advice - See the rubric on gradescope for additional details Thursday Nov 17, 2022: Peer-Review - Section 2 - Read section Peer-Review Process Tuesday Nov 22, 2022: Peer-Review - Section 1 - Read section Peer-Review Process (Thursday Nov 24, 2022): Thanksgiving Tuesday Nov 29, 2022: Guy and Jan; Nick - Section 1 - [Software engineering in start-up companies: An analysis of 88 experience reports (Springer)](https://link.springer.com/article/10.1007/s10664-018-9620-y) for Guy's and Jan's presentation - [Software Engineering for Machine Learning: A Case Study (IEEE)](https://ieeexplore.ieee.org/abstract/document/8804457) for Nick's presentation - Section 1 Friday Nov 30, 2022: Peer Review - Paper peer review due on [gradescope](https://www.gradescope.com/) Thursday Dec 1, 2022: Irmak and Sam - Section 2 - [Software Engineering for Machine Learning: A Case Study (IEEE)](https://ieeexplore.ieee.org/abstract/document/8804457) (Tuesday Dec 6, 2022): No presentation Wednesday Dec 7, 2022: Final Draft - Completed final draft due on [gradescope](https://www.gradescope.com/) (Thursday Dec 8, 2022): Reading day (Wednesday Dec 14, 2022): No Final # Using Hypothesis
# Presentation Requirements ## Before you present Here are some thing to consider before you present. We will discuss these items when we meet prior to your presentation. - Who is your audience? (peers, business, academics, investors, teaching, etc.) + Attire + Mannerisms + Formalism + Introductions + Motivation + How you engage - What methods will you use to calm your nerves? + Exercise + Breathing + Bathroom break - How will you prepare your space? + Laptop + Projector + Water - How will you reset your brain if you stumble? - What questions do you expect to receive? - How are you going to practice? - How long will you each present? ## Presentation rubric Here are the concepts on which I will provide feedback. - Presentation is clear and follows a coherent story. - Presentation media is easy to understand. - Teammates present equitably. - Presenters are poised and lively. - Presenters are prepared to answer questions. - Presentation duration is within the expected range (20 to 25 minutes). - Presentation includes three discussion questions. If you are looking for additional information, I recommend taking a look at these more formal/complete rubrics. - [Rubric for Formal Oral Communication](https://www.pomona.edu/administration/tlc/resources/rubric-formal-oral-communication) - [Oral Presentation Rubric](https://www.pomona.edu/administration/tlc/resources/oral-presentation-rubric) - [Multimedia Presentation Rubric | Pomona College in Claremont, California - Pomona College](https://www.pomona.edu/administration/tlc/resources/multimedia-presentation-rubric) ## After you present After your presentation, you will respond to the following prompts on gradescope. - Reflect on questions you received after and during your presentation. Were any unexpected? - What did you like about your presentation? - What would you do differently if you were to present the same topic again? # Presentation Advice - I do not have a preference for your presentation format. Common methods include: + Slides (including Pecha Kucha) + Live typing (similar content to slides) + Live diagraming + Live demos - I recommend listening to the [Think Fast, Talk Smart: The Podcast](https://www.gsb.stanford.edu/business-podcasts/think-fast-talk-smart-podcast). You might as well start week 1! - This is **not** a professional research talk. I would like to be informed, but it is also OK for you to also entertain. - It does not need to follow the same format/flow as the article(s) you present. - Presentations can come in any format. Here are a couple of great descriptions of how to craft a compelling presentation: + Simon Peyton Jones' advice on [How to give a great research talk](https://www.microsoft.com/en-us/research/academic-program/give-great-research-talk/). + [How to Make a Pecha Kucha - YouTube](https://www.youtube.com/watch?v=32WEzM3LFhw "How to Make a Pecha Kucha - YouTube") **I HIGHLY recommend watching the 7 minute How to Make a Pecha Kucha video above.** More information here [Pecha Kucha 20x20: Discover Pecha Kucha presentations, stories, ideas, examples, and videos that will inspire](https://www.pechakucha.com/ "Pecha Kucha 20x20: Discover Pecha Kucha presentations, stories, ideas, examples, and videos that will inspire"). # How to Find Articles I recommend the following approach for finding research papers: 1. Write a short list of search terms and synonyms. 2. Search for these terms using [Google Scholar](https://scholar.google.com/). - Look for "recent" papers (for deep learning I'd look for things published in the past two or three years) - Focus on papers with higher citation counts, but don't eliminate a paper just due to the number of citations - Look for "survey" and "review" articles, and then click on the "Cited by DDD" link to find recent papers that cite the survey/review article - If you find a particularly good article, you can also look at the papers it references (found at the end of the paper) 3. Search with the same terms using your search engine of choice - You can frequently find related blog posts or videos on the authors' websites 4. Use a tool like [Connected Papers](https://www.connectedpapers.com/) to find related articles. For other types of work (blog posts, etc.) use your normal searching skills (DDG, YouTube, etc.). # How to Read Articles Keep the following in mind when reading any research article: - **Who are the authors?** The authors are the only people with the full context needed to understand a paper. Many authors are not native English writers, so don't let lower-quality writing prevent you from understanding what could be an important takeaway. - **What is the venue?** Every venue will have its own requirements and expectations. Keep this in mind when thinking about what the authors left out (due to space requirements) or didn't include (due to the media). - **Who is the audience?** You are likely **not** the intended audience for most research papers. The intended audience is probably some niche sub-field of researchers from around the world. Every group of researchers has its own jargon and informal (not codified) rules and conventions. In general, I only gripe about grammar and style when **reviewing** (providing feedback prior to publication) an article, and not when reading an article or reviewing a research proposal. In the latter cases, do your best to look past writing characteristics (good or bad) and focus on contributions. Once you find a research article (or research heavy blog post), I recommend the following technique: 1. Skip the abstract 2. Read the first sentence of every paragraph in the paper 3. Come up with two or three questions you would ask the author 4. Summarize paper's main points in one to three sentences 5. Read the abstract and compare it to your summary 6. Read the paper again, but now without skipping any sections. In general, you only need to read in more detail if you need to understand something in more detail (for example, if you are trying to replicate their work). You should not expect to understand every part of a paper. You do not have the same background or context as the author(s). You will often want to read a paper several times if you are trying to recreate something the authors discovered. # General Writing Advice - Clearly state your thesis. - Provide clear evidence supporting your thesis. - Get into a writing routine. Write a small amount (20 to 30 minutes) three days a week for the entire semester. - Jot down disparate sentences and combine them together as is useful. - Outline sections headings and then write the first sentence of each paragraph for each section. # How to Write an Introduction You do not have to write your introduction in any specific format, but you might find Dr. Stirewalt's 5-paragraph rule for writing Introductions helpful: 1. **Introductory paragraph**: What is the problem and why is it relevant to the audience attending *THIS CONFERENCE*? Moreover, why is the problem hard, and what is your solution? You must be brief here. This forces you to boil down your contribution to its bare essence and communicate it directly. 2. **Background paragraph**: Elaborate on why the problem is hard, critically examining prior work, trying to tease out one or two central shortcomings that your solution overcomes. 3. **Transition paragraph**: What keen insight did you apply to overcome the shortcomings of other approaches? Structure this paragraph like a syllogism: Whereas $P$ and $P => Q$, infer $Q$. 4. **Details paragraph**: What technical challenges did you have to overcome and what kinds of validation did you perform? 5. **Assessment paragraph**: Assess your results and briefly state the broadly interesting conclusions that these results support. This may only take a couple of sentences. I usually then follow these sentences by an optional overview of the structure of the paper with interleaved section callouts. # Survey Paper Advice Most of you will decide to write a survey paper. So, what is a "survey paper"? > "A survey article ... should provide a comprehensive review of developments in a selected area, covering its development from its inception to its current state and beyond, and illustrating its development through liberal citations from the literature." > > -- [IEEE Communications Surveys & Tutorials](https://ieeexplore.ieee.org/xpl/aboutJournal.jsp?punumber=9739) > "A survey article assumes a general knowledge of the area; it emphasizes the classification of the existing literature, developing a perspective on the area, and evaluating trends." > "*Long Survey Paper:* A paper that summarizes and organizes recent research results in a novel way that integrates and adds understanding to work in the field. A survey article assumes a general knowledge of the area; it emphasizes the classification of the existing literature, developing a perspective on the area, and evaluating trends." > "*Short Survey paper:* Short survey papers are perspective pieces/essays that are well-focused and written by a leader in the field that usually presents a personal point of view critiquing widespread notions pertaining to a field. A perspective piece can be a review of a concept or a few related concepts." > "*Tutorial Paper:* A paper that organises and introduces work in the field. A tutorial paper assumes its audience is inexpert; it emphasizes the basic concepts of the field and provides concrete example that embody these concepts." > > -- [ACM Computing Surveys](https://dl.acm.org/journal/csur/editorial-charter) Here are two recent survey papers: - [A Review on Fairness in Machine Learning | ACM Computing Surveys](https://dl.acm.org/doi/10.1145/3494672). - [A Survey of Machine Learning for Computer Architecture and Systems | ACM Computing Surveys](https://dl.acm.org/doi/10.1145/3494523) I will look for the following in your survey paper: 1. Abstract (see [How to write a good abstract for a scientific paper or conference presentation - PMC](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3136027/)) 2. An introduction (see section section How to Write an Introduction) - Discuss the significance and topic motivation 3. A body including one to five sections (however you see fit to organize and discuss the topic) - Provide a view and categorization of existing work - Include your own commentary and assessments 4. A discussion (and possibly separate conclusion) - Provide your personal perspective on the topic - Include a discussion on future directions for the field Survey papers will often include definitions and figures copied from other papers (cited of course). As you read the articles you've accumulated, you should start organizing them into areas. The [Research Paper Rubric](https://www.pomona.edu/administration/tlc/resources/research-paper-rubric) will give you a good idea regarding how I will evaluate your report. If you want more information, then it won't hurt to skim through [Procedures for Performing Systematic Reviews](Kitchenham-2004-Systematicreviewsv4.pdf) by Barbara Kitchenham. # Peer-Review Process We'll perform peer reviews during two lecture periods. They will follow this format: 1. Read a peer's current draft. 2. Answer review prompts on gradescope. 3. Share your feedback with the author. 4. Repeat steps 1-3 with different peer some number of times. 5. Answer feedback prompts on gradescope. Note the following while reading your classmates' reports (you should take notes on gradescope). - What is their topic? - Did they outline their entire report? - Does the introduction help you understand the topic and its importance? - Does the introduction help you understand the topic's categorizations? - What did you find most confusing about their report? - What is a one sentence summary of their report? You can be brief in your note-taking. # Resources The [Center for Speaking, Writing, and the Image](https://www.pomona.edu/administration/writing-center) (formerly The Writing Center) is open this semester! They open at full capacity after the second week of the semester, but we will be holding limited appointments and drop-in hours as soon as classes begin. Writing and Speaking Partners meet one-on-one with students to talk about their work and provide feedback at any stage of their preparation process. Trained to think deeply about written, oral, and visual rhetoric and communication, these student peers facilitate conversations about everything from ID1 papers to senior theses, lab reports to creative writing, giving presentations to developing strategies for reading and engaging more deeply and confidently in class discussion. To make an appointment with a Writing or Speaking Partner, please - log onto the Portal and go to Academics > Writing Center, - contact them [by email](https://www.pomona.edu/administration/writing-center), or - drop-in during their hours hosted in SCC 148.