How does assessment work in PSYC3361?
This term you will work in teams to reproduce the descriptive statistics and plots from recently published replicationpapers. This is an opportunity to apply your new-found R skills and to learn about the challenges of open data and reproducible analysis. You will set your own challenge and then work together to complete it. You might not learn the exact functions that you will need to deal with your data problem in coding, so there will be a lot of TROUBLESHOOTING. But many hands make light work and learning R together works a little bit like these caterpillars.
A group of caterpillars, moving in a formation known as a rolling swarm. This rolling swarm of caterpillars moves faster than any single caterpillar. 🐛 pic.twitter.com/sn3J3CDYmE
— Theo Shantonas (@TheoShantonas) April 9, 2021
We will talk about what makes a team work well in class and this component of the course will be graded by peer and self assessment. You will rate your own and others contribution to the project using criteria that you decide on as a team.
In the first 3 weeks of term, you will work through online coding lessons that will set you up well to complete your reproducibility challenge. Danielle has set up a RStudioCloud workspace and posted YouTube videos talk you through the basics of RMarkdown, Data Visualisation, and Data Wrangling.
It will be your responsibility to watch them, follow along with the exercises and think hard about how the functions you are learning about might be useful in your project. From Week 4 you will start applying the code you are learning to your reproducibility challenge.
From Week 3, there will be two Q&A sessions each week. These are an opportunity to work together with your team in a computer lab, with course staff there to answer your questions. You need to attend at least one Q&A sessions each week. You will likely learn a lot both from asking about the problems you are having and from seeing the kinds of problems that other students are having.
For this component of the assessment, you need to attend Q&A and share a learning log to RPubs each week. The learning log should include a reflection of…
You can see an example learning log in this blog post. You should include example code/plots in your learning log as a way of sharing your progress with your team and Jenny/Kate/Danielle. Post a link to your RPubs learning to the slack channel by 11:59pm Sunday each week.
In Week 8, your group will present the outcome of your verification challenge to the class. You can choose whether you would like to present live or prerecord your talk. Each team member should be involved in the presentation. Your 10 min talk should include…
There will be time for Q&A after each talk.
You can find more detailed guidelines/tips in this document.
The goal of this section is to demonstrate that you have read and really thought about the paper. Write a short summary explaining the context of the research question, the methodology, the results, and the conclusions. Structure your summary around answering these questions; write in paragraphs (3 or 4 should be sufficient), not in Q and A format.
Then choose 3 of the following statements to guide your reaction to the paper. For each statement, write 3-4 sentences (150-200 words) explaining your point.
The goal of this section is to try and reproduce the “analysis”. For the purpose of this exercise, “analysis” includes the
WARNING: This task is unlikely to be easy but you will work on it in groups. In this section of your report you should document YOUR process and what YOU learned along the way. The writing doesn’t have to be scientific or formal; think of it as notes to your future self or explaining your code to a Rubber Duck (see https://en.wikipedia.org/wiki/Rubber_duck_debugging). Someone who has no idea what your code is trying to do should be able to read your notes and understand what the code is doing and why.
Open data is reusable and can be used to answer research questions that are different to those posed by the researchers who collected it. The goal of this section is allow you to “play” with the data and demonstrate some creativity and critical thinking skills. Now that you have tried to reproduce the analysis in the paper, what questions do you have about the data set? Use your data wrangling/plotting skills to ask and answer 3 additional questions.
The goal of this section is to give imaginary feedback to the authors of the paper you have tried to reproduce. Use the reproducibility checklist to make recommendations for things they could have done to make your job easier.
example verification report - you can use this report as a model for the style that you should be aiming for. You can see the Rmd file that produced the roaches report here. And a video of Jenny walking through the example here.