cwra+ data analytics · 6 cwra+ data analytics i levels of data there are three levels of data:...

1

Welcome to Data Analytics user guide. Data Analytics enables you to view your current and historical results and to create easy-to-share graphs. The platform will enable you to glean more meaning from your results and share those results more easily and more widely. You can use Data Analytics to view current or historical results going back to Fall 2013, examine trends across testing windows, and make reports that break down performance by gender, race/ethnicity, parental education, or English language status.

In addition to creating your own reports and graphs, Data Analytics is pre-loaded with a host of reports, including Total Scores, PT Scores, and SRQ Scores; PT and SRQ subscores; Time on PT/SRQ and PT/SRQ Scores; Effort and Engagement; and Mastery Level Distributions. Multiple graphs have already been generated from these reports and will auto-populate with your own school’s data when you log in.

The purpose of this user guide is to help you navigate the platform, with a large focus on generating your own graphs. You’ll find different sections on making a report, exploring reports, and how to make different types of graphs. You’ll also find a dedicated section explaining all the different features of our graphing system, some of which you might use more than others, or for some types of graphs but not others. We are always looking for feedback, so please don’t hesitate to contact us if you would like further help or explanation, or if you just want to tell us how we’re doing!

1

CWRA+ Data Analytics

HOW TO MAKE A REPORT

TYPES OF INPUTAfter logging in, click on Reports on the left-hand menu and choose New Report from the drop-down. On this page, there is a screen containing five types of inputs: Filtered Groups, Students, Variable Families, Variables, and Years. It is required to choose at least one Filtered Group, one Variable, and one Year, in order to make a report. To choose a Filtered Group, click the left-facing arrow next to Filtered Groups to reveal an input area.

2

CWRA+ Data Analytics Guide

Filtered Groups corresponds to rows of the data set. For a data set to contain every student available, select My Students.

To make a report that aggregates results between different demographic groups, select one or more of those groups. For instance, to make a report that shows average Total Score for males and for females, select both Male Students and Female Students. To make a report that shows average Total Score for just female students, select only Female Students. For all students, choose My Students. This corresponds to the students at your institution.

Click the same arrow, now facing down, to reveal the Filtered Groups input area to hide it again.

The next input is Students. Clicking on the left-facing arrow will give the option of selecting individual students. This is only useful for reports that focus on one or a few individual students instead of entire classes of students.

3


Variable Families simply corresponds to groups of similar Variables. For instance, choosing Performance Task will automatically add the three PT subscores, the overall PT score, the effort and engagement on PT questions, and time on PT to a report. This is a useful shortcut for adding many outcome variables at once. Otherwise, individual variables can be chosen in the next input area. Note that variable families are not selected here.

The Variables input allows the user to choose one or more individual outcome variables. Here, the Total Score, Performance Task Score, and Selected-Response Questions Score variables are selected.

4


Finally, Years allows the user to specify which academic years to view. Data Analytics shows data from fall 2013 onward, which is when CWRA became CWRA+. Fall and Spring testing windows are not differentiated in Data Analytics, so the time variable is not winnowed down any further than the entire academic year. Here, all four academic years are selected. Once the first academic year is selected, the Run Report button becomes visible.

EXPLORING YOUR REPORT

EDITING YOUR REPORTOnce a report generates, the first thing to note is that there is an Edit Inputs tab at the top of the page.

Clicking the Edit Inputs tab will bring you back to the report creation window. This will allow you to edit how the report is made and rerun it. You can also use this function to review the inputs used to make the report.

Save the report before doing anything else.

5


Clicking on the Save button will require you to provide a Title. You may also give an optional description. Click Save.

On the left-hand menu, there will be an entry under Open Reports corresponding to the report you are currently working on. Clicking on Saved Reports also shows this report listed among your saved reports. The Info tab in your report contains the information you just input: Title, Description, and any other relevant information.

6


LEVELS OF DATAThere are three levels of data: Class, Institution, and Student.

Class-level data: Class-level data shows you mean scores for the outcome variables you selected for each class level in each academic year that appears in your data set.

For instance, here, the data set has Freshmen, Sophomores and Seniors from 2013-14; Freshmen, Sophomores, Juniors, and Seniors from 2014-2015; etc. The columns can be resized by clicking and dragging the vertical lines between columns. You can also see more score information, such as counts of scores and minimum and maximum scores, by clicking on the little icon that looks like three horizontal linestriple bar icon and selecting options from the window that pops up.

You can also click and drag on column headers to reorder the columns. If you select Male Students and Female Students in the Filtered Groups portion of the report generation, there would be two for each class level and each academic window: one for the male students and another for the female students.

7


Institution-level Data: Institution-level data pertains to value-added scores. If you test freshmen and seniors in the same academic year, you will have a value-added score displayed here as long as you selected at least one of Total Score, PT Score, or SRQ Score as outcome variables. Note that Data Analytics does not require that freshmen test in the fall and seniors test in the spring in order to obtain value-added scores, although our traditional PDF reports do still have this requirement.

Student-level Data: Student-level data allows you to see results for individual students.

In this guide, the students’ names are obscured to maintain privacy, but in your own reports you would see your individual students’ names. Remember, you can only see students and their data for your own school. You cannot see another school’s data, even with the names and other identifying information removed.

You can download the data as a CSV file, which can be easily imported into Excel or opened as a plain-text file, by clicking on the Download button and choosing which level of data you want to download: Class, Institution, or Student.

8


HOW GRAPHING WORKS

ADDING A CHARTTo add a chart, click on Add Custom Chart on the right-most tab in your report. Then, click on New Chart.

A pop-up window with two tabs will appear.

9


SETTINGSThe first tab, Settings, allows you to give your graph a Title and a Description. You can also choose Scaling options, which simply correspond to whether Data Analytics should zoom out further on the y-axis than whatever the default behavior is. Since it is unknown how it will default at this point, leave this drop-down menu untouched. Featured corresponds to whether or not this graph appears in a gallery of graphs on the home screen, as a sort of short cut that you can quickly jump back into. There is also a link below to delete this chart, if you change your mind about making it or simply don’t like what it looks like.

DATA SERIESClick on the Series 1 tab.

In the Series 1 tab you can specify the details of what exactly you will be graphing. Because the tool is flexible, you will have to be specific in your graphic specs.

Filtered GroupFirst, choose a Filtered Group from among those that were input into the report.

If your report only contains My Students or All Students – that is, no further demographic breakdown – then that is your only choice. You must choose a group here, even if it is just My Students. However, if you had selected some demographic filtered groups in your report, you can select one of those groups to graph separately. For instance, to make a report comparing males vs. females, graph each group as its own data series separately.

10


StudentsIn the next section, Students, you must select a class level to graph. Clicking the page right button displays the remaining classes not already visible, and then also displays individual students. There are two main considerations here. First, is which class level to select. Remember, you are able to add more data series onto the same chart afterwards, so this is only about which class level you want to plot in the current data series. The second consideration is whether you are adding a data series that shows a data point for each individual student in the class, such as you would do on a scatter plot where each student in the class gets his or her own point; or adding a data series that summarizes across all students in the class, such as you would do on a bar or line graph where a “point” on the graph summarizes across all students in that class. If you are adding a data series that shows a data point for each student in that class, select one of the “Each” class level options. If you are adding a data series that summarizes across all students in that class, select one of the “All” class level options. For instance, to make a scatter plot showing PT Score vs. SRQ Score for freshmen, select Each Freshman, so that each freshman would have their own data point. To make a bar graph showing mean PT score at every class level, start by selecting All Freshmen, because the bar corresponds to the aggregate across all students in that class. Then add another data series for every other class level that tested (e.g., All Sophomores, All Juniors, and All Seniors, for whichever class level(s) tested).

YearNext, choose a Year. If you only chose one Year when making your report, select that year. Otherwise, select among the years you input into the report. Alternately, if you want your graph to show change over time, you can select Each Year instead of an individual year. In that case, points on the x-axis will correspond to academic years.

The Y-Axis should be whichever variable goes on your vertical axis. This is usually some type of score or subscore, time on task, or effort or engagement response (i.e., something quantitative). The X-Axis should be whichever variable goes on your horizontal axis. It can be My Students, which will allow you to put class levels on the x-axis side-by-side. If you chose Each Year in the Year drop-down, you can select Years to make sure that the graph properly labels the consecutive academic years on the x-axis. If you are going to put filtered groups side-by-side, such as males vs. females, then choose the Comparison Groups option.

Optional FieldsClick on show optional fields to reveal two more options.

Here you can choose the type of graph you want to create if this is the first data series that you’re adding to the graph. If this is not the first data series because you’ve already added one, you shouldn’t change the chart type that’s already selected. You can also choose a specific color for the current data series, although if you leave this blank you will automatically get a different color for each data series.

• Once all options are specified, do one of the following: • If this is the first data series you’re adding to the graph (i.e., the graph has not yet been created), then click on

the Add Data to Current Series button. • If you’ve already created the graph but you wanted to change something about the data series, click on the

Update Series Settings button.• If you already created the graph but, instead of changing something about the current series, you are making

a second series to add to the graph, click on the Add Data to Current Series button.

11


DIFFERENT TYPES OF GRAPHS

With CWRA+ Data Analytics, you have the ability to create bar, line, scatter, and pie graphs.

This guide will use specific reports to demonstrate how to make these graphs. The first set of graphs are based on a report already generated that shows the three PT subscores (APS, WE, and WM) for all four academic years since the introduction of CWRA+ (2013/14 through 2016/17). If you need assistance making graphs based on other reports, please contact your CWRA+ Rep.

BAR GRAPHSFirst, create a bar graph that shows the mean APS subscore obtained by all students at this institution in the 2014/15 academic year. Click on Add Custom Chart.

12


Give the graph the following title and description, and choose to allow the graph to be displayed on our home page by making it featured.

1. Then click on the Series 1 tab. 2. Choose My Students or All Students for Filtered Group.3. In the Students section, choose All Freshmen by clicking on the circle button to the right. Choose All

Freshmen instead of Each Freshman because, for a bar graph, each bar summarizes information across all students in that class.

4. In the Year section, choose Academic Year-2014/15, as previously specified.5. For Y-axis, choose Analysis and Problem Solving subscore, since the height of each bar should to correspond

to the mean of this variable (and thus leave Average selected in the drop-down menu that appears to the right of the y-axis drop-down menu).

6. For X-axis, choose Default Labels. By default, CLA+ Data Analytics will show you class level as the x-axis label when nothing different is specified.

7. Then, click on show optional fields. Choose Vertical Bar as the chart type. This corresponds to a vertical bar plot.

8. Finally, click Add Data to Current Series.

13


You will see a bar graph created in the background, behind the graphing tab you were just working on. To add another data series for the next class level, click on the circle button next to All Sophomores to swap this class level in for freshmen.

Then click Add Data to Current Series to add this is as a new data series. Note: Clicking Add Data to Current Series will add this series to the current one. To replace the current data series into a new data series, click Update Series Settings.

14


After doing this, choose All Juniors and click Add Data to Current Series again. Finally, using the right arrow just below the All Juniors option, go to the second page of class levels and choose All Seniors and click Add Data to Current Series for the final time. Then close the graphing tab to reveal the final graph.

LINE GRAPHSNext, make a line graph illustrating the change in WE subscores over time for our seniors. Click on Add Custom Chart again and give the graph the following title and description.

15


Click on the Series 1 tab to enter more graphing information.

1. Choose My Students/All Students in the Filtered Group area.2. Choose All Seniors below that to summarize information across all seniors in each point on the line graph. 3. Select Each Year in the Year drop-down.4. In the Y-Axis drop-down, choose Writing Effectiveness subscore.5. In the X-Axis drop-down, choose Years. 6. Click on show optional fields and then select Line in the Chart Type drop-down.7. Click Add to Current Data Series.

16


Close the graphing pop-up window to reveal the line graph we just created.

This looks like a pretty drastic change in Writing Effectiveness subscore, but notice that the y-axis only scales from 2.875 to 3.025. This is because Data Analytics automatically zooms in to minimize white space in the graph. However, in this case, the actual change in score relative to possible change in score is distorted. Click on the red gears button to bring the graphing window back up.

In the Settings tab, choose “100% larger” in the scaling drop-down. The graph is automatically adjusted, so there’s no Update or Save button to click. Just close out of the tab again.

On a scale more representative of the true range of scores, the decline in WE subscores over time is almost flat.

17


PIE CHARTNext, we will show the distribution of Writing Mechanics subscores.

Click on Add Custom Chart again and fill in the following information.

Click on the Series 1 tab to enter in more graphing information.

1. Choose My Students/All Students from the Filtered Group drop-down menu. 2. This graph will only include seniors, so choose Each Senior (because we are not going to average over scores,

but instead make a graph that shows the distribution of each senior’s WM subscore). 3. In the Year field, choose Academic Year-2016/17. 4. Choose Writing Mechanics subscores for X-Axis.5. Choose Distribution of x-values for Y-Axis. (Although this will be a pie chart and x- and y-axis may be

difficult to think of in this context, keep in mind that any distribution of scores shows possible values on the horizontal axis and frequency of those values on the vertical.)

6. Click on show optional fields. 7. In the optional fields, choose Pie from the Chart Type drop-down. Then click Add Data to Current Series.

18


The resulting pie chart shows a wedge for each observed WM value (integers only) in the data. By hovering your mouse over each section (not shown), you can get a pop-up window that tells you which WM subscore corresponds to each wedge of the pie chart. Scores of 0 are actually unscorable responses, not the absence of WM skills. There are a few unscorable responses in this data set. Also, there are no perfect scores (6s), but every other score is represented.

19


SCATTER PLOTSThe next set of graphs are based on a report where male and female students are chosen as the filtered groups. PT Score and Time on PT are chosen as the variables.

Click on Add Custom Chart.

Fill in the following information.

Click on the Series 1 tab to fill out the following graphing information.

1. Choose Male Students in the Filtered Group field (you can only plot one filtered group at a time). 2. In the Students field, choose Each Senior.3. For the Year, choose Academic Year – 2016/17.4. Choose Performance Task score in the Y-Axis drop-down.5. Choose Total time spent on the PT in the X-Axis drop-down.6. Click on show optional fields and choose Scatter in the Chart Type field. 7. Click Add Data to Current Series.

20


This scatter plot shows time on PT vs. PT score for male seniors in 2016/17. Click on the red gears icon to bring up the graphing window again.

21


Now change the Filtered Group from Male Students to Female Students. Click Add Data to Current Series to add this as a new data series to the same chart.

Although a bit difficult to read and interpret, you can very easily plot the relationship between time on PT and PT score for each gender superimposed onto one graph.

cwra+ data analytics · 6 cwra+ data analytics i levels of data there are three levels of data:...

Documents