# Project 1: Table Read! ## Due Date Information **Out:** Monday, October 5th **Design Checks:** Tuesday, October 13th - Wednesday, October 14th Your design check TA should be reaching out by email soon to schedule a design check. Remember to include your project partner in the Google Calendar invite! Your design check plan is due at 10PM EST the day before your design check. **In:** Friday, October 23rd, 11:59PM EST!  *[Image source](https://www.htmarket.com/ht-reel-dct.html)* ## Summary It's time to try your data science skills on real datasets! For this project, we will be looking at data on movies as provided by [Rotten Tomatoes](https://www.rottentomatoes.com/). The project occurs in **two stages**: in the first week, you'll work on planning out your analysis and meet with a TA during a **Design Check** to get feedback on your plan. In the second week, you'll implement your plan, produce your results, and present them in a meeting with your TA. This is a pair project. You can find your partner assignment as a pinned post on Ed. Note that you have to work with different partners on the first two projects in the course (the last project is a solo project). :::info We suggest skimming the *whole* handout to get an idea of what is expected, then reading the design check instructions, and then opening up the data set to explore it in more detail. The overarching goal of this project is to answer the analysis questions that we provide for you and that you provide yourself. Do not worry if you and your partner do not immediately arrive at the list of tasks necessary to complete the project -- one of the skills you are practicing is how to break a large analysis down into smaller steps. The design check with your TA will be one opportunity to let you know if you are on the right track. ::: ## Resources - [CSCI0111 Table Documentation](https://cs111-docs.cs.brown.edu/s/table) - [Pyret String documentation](https://www.pyret.org/docs/latest/strings.html) - [Edstem](https://edstem.org/us/courses/102199/discussion) - [Code style guide](https://cs111-docs.cs.brown.edu/s/pyret-guide#) ## What Skills Does this Project Practice? These high-level descriptions highlight the skills that you'll practice in this project: - preparing tables for use by cleaning up or detecting messy data - combining data across tables to answer a question - using plots and charts to display data - creating functions/helper functions to reuse common computations We have done every one of these steps across lecture, homework 3, and labs 3 and 4. You have a lot to start from. ## The Data The [stencil code](https://code.pyret.org/editor#share=1F2HLk-tzSqVM_JaoySVsi2zwKtqiFcYS&v=99a1e4d) will load all of the tables that you need. Make a copy of it now. You have two main tables available to you. `movies` provides information on various movies released 2020-2023. `reviews` lists a subset of the reviews some of these movies received. **Take a moment now to view these tables in Pyret.** :::warning **Note:** the original reviews dataset had almost 89k rows! To make loading in the data a bit easier, we pared down this table to around 20k rows. This means that this isn't the full dataset that Rotten Tomatoes provides, so your statistical analysis won't necessarily be accurate to the original dataset. That's okay; we're looking to see what conclusions you draw based on data from the provided tables! ::: You also have access to three "helper tables," each of which has one column: `years`, `genres`, and `languages`. These provide the unique values from the dataset for the corresponding information. Your analysis will require at least one of these tables, so you should uncomment the relevant block of code to load it in. One of your first implementation tasks will be to clean the data. Pay special attention to what cleaning may be necessary, especially for columns that link information across tables. --- ## Phase 1: The Design Stage :::danger Every semester, we get students who thought they needed to have all of the code for the project written before the design check. **You don't!** The point of the design check is for you and your partner to understand what the goal of the project is, to propose analysis questions, to make a plan for the coding, and to get a chance to course-correct with a TA and ask any clarifying questions *before* you dive in. ::: The design check is a 30-minute meeting between your team and a TA to review your project plans, clarify conceptual misunderstandings, and give you feedback before the deadline (Asking questions is encouraged!). You do **not** have to have finished (or even started) coding up the project before the design check! Many students make changes to their designs following the check: doing so is common and will not cost you points, and might even save you some work when you do start coding. Before your design check, you will turn in a document with your work for the following tasks: **Design Task 1 -- Understand Your Data:** Use the stencil code to load the data set into Pyret. Look at the structure and contents of each provided table. In one place, create a reference sheet that you can refer to for the rest of the project and bring to the design check. Put the following on the reference sheet: 1. The *exact* names of each of the provided tables 2. The *exact* column names of each of the provided tables 3. One (or more!) example rows from each of the provided tables 4. Some representation of the relationships/common information between the tables. For example, if two different tables have columns with the same information, it may help to draw an arrow or use color-coded highlighting to show that on your document You can also include any other information/notes that you find useful, but don't crowd the reference sheet too much. Ideally, it should fit on one or two pages so that it can be a useful at-a-glance guide. **Making a reference sheet like this will save you time as you consider your analysis, make a programming plan, and start coding!** You should plan to bring your reference sheet to any office hours that you attend for this project. **Design Task 2 -- Data Cleaning Plan:** Look at your datasets and make a to-do list of the cleaning, normalization, and other pre-processing steps that will need to happen to prepare your dataset for use. (*Hint: look for similar data in different tables that has different formats. Also think about whether data in some of the columns *) **A note on the dates:** you can assume that any date is in the format year, month, day OR month, day, year. Because the movies in the dataset all came out in 2020-2023, any date in the dataset can therefore be unambiguously determined to be one of these formats. :::success After the design check, we will provide a list of required data cleaning tasks for the assignment. You won't miss points in the design check if you didn't think of these tasks, but you will be responsible for implementing them for the final submission. For example, one required task will be to make a version of `movies` that removes any movie without a review. **The required data cleaning tasks are [here](https://edstem.org/us/courses/102199/discussion/8397368)** ::: **Design Task 3 -- Proposed analysis** You will start off by answering the question **"Is there any correlation between box office revenue and audience score for the movies?"**, but the rest of the analysis is up to you! Propose **3 more analysis questions** that ask questions you're interested in figuring out for this dataset! There are a *lot* of options here, but you must heed the following: - At least one question has to *group* the movies according to year, genre, or language (i.e. to answer this question, you will probably need to add a column to the `years`, `genres`, or `languages` table). - At least two questions must have something to do with the reviews. - The answers to your questions must make use of at least two different operations from the [summarizing columns](https://cs111-docs.cs.brown.edu/s/table#Summarizing-Columns) section of the Tables documentation (for example, you might take the average of a column to answer one question, and you might sum up a column to answer a different question). - There must be a compelling way to present the results of your analysis for each question (such as a single plot, or a single *small, human-readable* table). In your final submission, you will be assessed on your choice of presentation. Make your questions compelling enough to introduce some complexity, but don't create too much work for yourself by designing a very contrived question. If you find yourself asking about a lot of things in one question, break it up! You'll get some feedback on the complexity of your questions in the design check. **Design Task 4 -- Analysis Plan:** For each of the four analysis questions (including the one we provided), describe how you plan to do the analysis. You should try to answer these questions: * What charts, plots and statistics do you plan to generate to answer the analysis questions? Why? What are the types and the axes of these charts, plots and statistics? * What table(s) will you need to generate those charts, plots and statistics? Try to give exact column names -- the more specific you are, the better the feedback the TA can give. If you'll need to build columns or construct additional tables, make note! * [Plan out](https://cs111-docs.cs.brown.edu/s/planning) the coding tasks that you will need. **Especially do this** for any new tables you plan to create! **Design Task 5 -- Partner Agreement:** Have in writing an agreement for how you and your partner will work on the implementation (see the "working with your partner" section at the end of the handout). You can have this in email, but you will need to show something written to your Design Check TA. ### Design Check Handin ++By 10pm EST the day before++ your design check starts, submit your work for the design check as a PDF file to "Project 1 Design Check" on Gradescope. Please add your project partner to your submission on Gradescope as well so that there is one submission with both of your names attached to it. You can create a PDF by writing in your favorite word processor (Word, Google Docs, etc) then saving or exporting to PDF. Ask the TAs if you need help with this. Please put both you and your partner's CS login information at the top of the file. ### Design Check Logistics * Please bring your work for the design check either on laptop/tablet (files already open and ready to go) or as a printout. Use whichever format you will find it easier to take notes on. * We expect that both partners have equally participated in designing the project. The TA may ask either one of you to answer questions about the work you present. Splitting the work such that each of you does 1-2 of the analysis questions is likely to backfire, as you might have inconsistent tables and insufficient understanding of the work done by your partner. * Be on time to your design check. If one partner is sick, contact the TA and try to reschedule rather than have only one person do the design check. ### Design Check Grading The design check is graded on **good-effort completion,** meaning that you turned in complete and reasonable responses to each of the five tasks above and were able to explain them thoughtfully to the TA. For example, for each analysis question you propose, we expect you to be able to describe why you picked a particular plot or table format, but we won't penalize you for picking a plot that isn't effective in the context of the question. Your TA will give feedback for you to consider in your final implementation of the project. **Everyone who puts in a good-faith effort towards the design check gets 100% on it.** Your design check grade will be worth roughly a third of your overall project grade. Failure to account for key design feedback in your final solution may result in a deduction on your analysis stage grade, especially if the feedback asked you to add complexity into your analysis questions. ***Note:** We believe the hardest part of this assignment lies in figuring out what analyses you will do and in creating the tables you need for those analyses. Once you have created the tables, the remaining code should be similar to what you have written for homework and lab. Take the Design Check seriously. Plan enough time to think out your table and analysis designs, and ask questions during the design check!* --- ## Phase 2: Implementation and Reporting In phase 2, it's time to write code to complete your analysis! After you hand in the project, you will **defend your conclusions to your design check TA** by explaining how you wrote code to generate your analysis artifacts (plots/tables) and how these artifacts answer your analysis questions. ### Implementation Broadly, your phase 2 implementation work is geared towards writing code that generates the analysis artifacts you proposed in your design check (or decided to generate after receiving design check feedback). Start with the cleaning tasks, and then implement your plan from the design check. You will need to keep track of multiple tables here, so this is where your summary sheet from the first design step task will come in handy! Feel free to add any helper function descriptions (and especially the tables they're meant to apply to) to that sheet as well. **Hint:** If you feel your code is getting too complicated to test, add helper functions! You will almostly certainly have computations that get done multiple times with different data for this problem. Create and test a helper or two to keep the problem manageable. You don't need helpers for everything, though -- for example, it is fine for you to have nested `build-column` expressions in your solution. Don't hesitate to reach out to us if you want to review your ideas for breaking down this problem. **Final note:** Using any function that takes in or outputs Lists for this assignment is NOT allowed, even if they are in the Tables documentation. [Pyret String documentation](https://www.pyret.org/docs/latest/strings.html) might be your friend! :::info **Code running slowly?** The tables you are working with are very big, so if you are doing a lot of operations, the code will take a while to run. To save time, we suggest starting with a new Pyret file with smaller example tables while coding and testing your analysis, and then moving your code over and adapting it to work with the larger tables once you're confident with your analysis. Think about how functions can help you accomplish this task! This is also how code that works on large datasets is developed in industry. ::: ### Handin Prior to your grading meeting, you will hand in the following: 1. A Pyret file named `analysis.arr` that contains all the functions used to generate the analysis. Nothing is required to print in the interactions window when we run your analysis file. 2. A file named `report.pdf` that lists each of your four analysis questions and the artifact used to answer them. You should also include some brief notes (can be a bulleted list) of how the artifact answers the question and connect your code to your analysis (see below). **Note:** Pyret makes it easy to extract image files of plots to put into your report. When you make a plot, there is an option in the top left hand side of the window to save the chart as a `.png` file which you can then copy into your document. Additionally, whenever you output a table in the interactions window, Pyret gives you the option to copy the table. If you copy the table into some spreadsheet, it will be formatted as a table that you can then copy into Word or Google Docs. **Note:** Please connect the code in your `analysis` file and the results in your `report` with specific comments in the code and notes in the report. <details> <summary> An example of how to link the code file and report</summary> ***Code file: includes comment*** ``` # Analysis for question 1 (on correlation between revenue and audience score) fun plot-q1(movie-tab :: Table) -> Image: # helper function to make a plot ... end plot-q1(cleaned-movies) # call of the plotting function ``` Under Question 1 of the report, indicate that the expression `plot-q1(cleaned-movies)` is used to generate the plot you presented. </details> ### Grading meeting After your handin, you will meet with your TA again to defend your analysis. Broadly, the questions you will be asked are: - How does the artifact you chose support the conclusion you drew for each question in the report? - Can you explain how you wrote the code to generate each artifact? - How did you go about setting up smaller tables for testing functions before using them on large datasheets? - Can you give specific examples of how you used helper functions to reduce repetition in your code? Your TA will also review your code and report to answer the following functionality, testing, and design questions **Functionality:** Were you able to use code to perform the table cleaning tasks we required? **Testing:** Have you tested your functions well, particularly those that do computations more interesting than extracting cells and comparing them to other values? **Design:** - Did your analysis questions meet all of the guidelines we asked for in the design check? - Have you chosen appropriate functions and operations to perform your computations? - Have you used docstrings and comments to effectively explain your code to others? - Hve you named intermediate computations and helper functions appropriately to improve readability of your code? This includes both what you named and whether the names are sufficiently descriptive to convey useful information about your computation. - Have you followed the other guidelines of the style guide (line length, naming convention, type annotations, etc.) As you can see, the majority of the metrics we're measuring have to do with *design*, because this is an open-ended and ambitious project that required you to carefully consider how you structured your approach. The difference between high and mid-range scores will lie in whether your computations that create additional tables are clear and well-structured, rather than appearing as you made some messy choices just to get things to work. --- ## Working with Your Partner We expect that both partners are involved in the work of this project. Specifically, this means: - You do the design work together and present your ideas to your design-check TA together. - You cooperate on the implementation. There are multiple ways to do this: - Write the code for both parts working mostly together - Each work on individual functions separately, but while working in proximity (same room, online together, etc) - Each write some parts and check in periodically to agree on the code that gets submitted How you arrange your work is up to the two of you. As part of your design check, you will indicate how you plan to do the implementation work. **Be respectful of each other's time**. If you agree to meet to work on the project, show up as scheduled. If you agreed to get certain work started prior to a meeting, come with that work started. This is basic professionalism. **What if a partner stops responding?** Get in touch with your design check TA, Milda, and the HTAs if your partner becomes unresponsive, whether that means they are not doing their share or they are doing the work alone and leaving you out of it. Neither is acceptable. **You will only get credit for a project that you actively participated in**. If you left your partner to do all of the implementation work, you will not get credit for that portion of the project. ---------------------- > Brown University CSCI 0111 (Fall 2026) > Do you have feedback? Fill out [this form](https://forms.gle/uJhBwq4ocmQ6vEq89). > <iframe src="https://docs.google.com/forms/d/e/1FAIpQLSes93TjDZdG7wLuchLdO8E27OJkfE5Uujz3aKoz7bivOFY19A/viewform?embedded=true" width="640" height="982" frameborder="0" marginheight="0" marginwidth="0">Loading…</iframe>