* The pilot helped confirmed that the topic of news framing can be studied using a textual analysis software such as Voyant.
* All students managed to use Voyant and generate data outputs as required.
* Students incorporated Voyant's visual outputs into the paper and referenced the visuals within their papers with varying degrees of success.
* Students demonstrated a superficial understanding of the analysis by relying heavily on the cirrus output rather than both the trend and cirrus outputs. More time will be dedicated to distinguishing the differences in the Spring. Assignment description will need to provide a clearer expectation.
* Tech fellow's support was helpful but only three students took advantage of the opportunity.
* The three scaffolded exercises were helpful in preparing the students for the final framing paper.
* For the Spring semester, one of the exercises will be changed so it's tied directly with the Voyant analyses.
* Classroom instructions though helpful, the plan in the spring is to create a video tutorial as a supplemental material for students to use outside of the class.

The students learned the basic use of Voyant and had a solid grasp of the framing theory. They quite successfully identified the various news frames used at the national and state level, but when it comes to the thoughtful and appropriate use of Voyant analysis and output, there is still room for improvement. The average grade this class received on being able to use appropriate and thoughtful Voyant analysis as supporting evidence for their proposed frames was a D. Asking them to compare two datasets (national and state-level) on top of expecting them to retrieve their own datasets might have made the process and steps a bit too complex. The next time I use this assignment, I would: 1. Scale it back down. 2. Narrow down the research questions even further. 3. For a theory class (as opposed to a methods class), it'll save time and avoid confusion to provide the dataset instead of asking the students to retrieve and clean them. At least provide the larger dataset and have them locate the small samples.

Considering this being my fourth semester implementing this assignment, I do think the outcome ended closest to what I've expected. Overall, students were able to toggle between the close reading of the news stories and the distant reading based on the Voyant analysis. This class demonstrated a stronger ability to draw inferences from both the qualitative quotes and the quantitative outputs. The majority of the students were successful at identifying three distinct news frames based on the four human interest stories they were required to analyze. Students varied in ability when it comes to using relevant qualitative and quantitative evidence to support and develop the observed frames. The class average for the 6-7 page paper was 80%. This is an increase from the average of 69% in the previous semester. When breaking down the rubric to only look at student's performance in using Voyant for the quantitative analysis, this class averaged 75%. It's an improvement from last semester's average of 60%. Aside from the many other factors, I think one big difference between students’ performance this semester compare to last has to do with me scaling the sample size back down from 100+ to four. Consider that CMM306 is NOT a methods course, trying to take on a project that truly deals with big data but still trying to cover other theoretical content is too big of a bite to take. The take away here is that the learning goals should decide the size of the data. For a theory course like CMM306, the small sample size is more suitable. The next time I use this assignment in a theory course like CMM306, I will continue to use the scaled down version—I might scale down further. Instead of just scaling down in sample size, I would also narrow the scope of the research questions. I already did so this semester, but I think there is room to narrow further, especially since the topic of the opioid crisis has developed into a multifaceted issue.