Suggestion Box

Spot an error or have suggestions for improvement on these notes? Let us know!

Week 11 · Scoring, Filtering, and Summarizing One Participant

0 · Overview

Before scoring and filtering data, it is important to first import and inspect a participant file to understand its structure. This section focuses on transforming raw data into usable variables for analysis.

We will learn to:

Score a questionnaire that is stored as JSON.
Filter implausible reaction times before computing performance metrics.
Compute mean reaction time (RT) and accuracy for the practice and experiment blocks.

The goal is to turn raw data into valid, interpretable measures that summarize each participant’s behavior and responses.

1 · What “Scoring” Means in This Context

In most experiments, we collect two main types of data:

Behavioral performance, such as reaction times and accuracy, and
Questionnaire responses, which measure self-reported states or attitudes.

Questionnaire responses need to be converted into a single number that represents a participant’s overall score on a construct, such as engagement or focus. This step is called scoring.
Some items are worded in the opposite direction and must be reverse scored so that higher numbers always mean more of the same construct.

This is a common workflow in psychological experiments, where we pair self-report data with behavioral measures to better understand both subjective and objective aspects of performance.

2 · Example Questionnaire Aligned to the Task

Task Engagement and Focus (TEF-10)
Scale used by jsPsychSurveyLikert with default numeric values:

0 = Strongly Disagree
1 = Disagree
2 = Neutral
3 = Agree
4 = Strongly Agree

Items (completed after the N&P task):

I stayed focused on the digits.
I felt distracted during the task. (reverse)
I understood what each key meant.
My mind wandered while I was responding. (reverse)
I tried my best throughout the task.
I felt confident in my responses.
I rushed through the trials without thinking. (reverse)
The instructions were clear.
I paid attention to the mapping rules.
I was motivated to do well.

Reverse-scored items: 2, 4, 7

Because jsPsychSurveyLikert starts at 0 and ends at 4, the reverse-score formula is:

reversed_value = 4 − original_value

If your questionnaire instead used a 1–5 scale, the reverse formula would be:
reversed_value = 6 − original_value

Understanding the structure of this questionnaire will help you write the scoring function in a systematic way. Each step of the code will mirror what you would do conceptually: extract responses, correct the reversed ones, and then calculate a total or mean score.

3 · Building the Questionnaire-Scoring Function

The next step is to organize your code into modular scripts that can be reused and debugged easily. Rather than writing everything directly in the Quarto report, we will create standalone R scripts that contain specific functions. The first script you will create is called score_questionnaire.R.

Remember that the questionnaire responses are stored in a JSON string in the response column, which looks something like: {"item1":4, "item2":3, "item3":2, ...}

You will now create a function in scripts/score_questionnaire.R.
This function will take the JSON string from the response column and return a single numeric score for that participant.

The commented structure below outlines the steps. Read through it and think about what each line is doing before filling it in.

#### score_questionnaire.R -----------------------------------------------------
## Purpose: Take a JSON string from the questionnaire row and return a single score.
## Scale: jsPsychSurveyLikert default 0–4. Reverse items: 2, 4, 7.

## 1) Parse the JSON string into an R object
##    Use jsonlite::fromJSON() to convert the text into a list.

## 2) Flatten and convert to numeric
##    Use unlist() to turn the list into a vector and coerce to numeric if needed.

## 3) Validate responses
##    Check that all values are within 0–4 and handle missing values (NA) if they appear.

## 4) Reverse-score the specified items
## Example:
## responses[rev_items] <- 4 - responses[rev_items]
##
##    Identify which items should be reversed (2, 4, 7) and apply the formula above.

## 5) Compute the final score
## Example:
## mean_score <- mean(responses, na.rm = TRUE)
##
##    You can choose to use mean() or sum() depending on how the scale is defined.
##    Return a single numeric value representing the participant’s score.

Things to check as you go:

What happens if the JSON string is missing?
Are any values outside 0–4?
How should missing items be handled?

When finished, test the function on one participant’s data to confirm it returns a sensible score. Then clear your environment and use source("scripts/score_questionnaire.R") to confirm it loads cleanly.

Once the questionnaire has been scored, we can turn to the task performance data.
These two components—self-report and behavioral performance—will eventually be combined to form each participant’s summary row.
Before we can compute those summaries, we need to clean the reaction time data by removing implausible responses.

4 · Filtering Reaction Times Before Summarizing

Reaction time (RT) data often contain outliers.
Some responses are unrealistically fast (accidental keypresses), while others are extremely slow (distraction or inattention).
Filtering helps remove those implausible values before calculating averages.

For this task, we will exclude:

RTs below 250 ms (too fast)
RTs above 900 ms (too slow)

Filtering should be done separately within each block to avoid removing trials incorrectly.

Typical workflow:

Split the data into practice and experiment blocks.
Within the experiment block, split by trial type (for example, magnitude vs parity).
Filter each subset to keep only RTs between 250 and 900 ms.
Compute mean RT and accuracy for each subset.

This process ensures that your summaries reflect valid responses only.

filtered_data <- data[data$rt >= 250 & data$rt <= 900, ]

After filtering, check that the number of trials looks reasonable and that the averages have not changed in unexpected ways.

5 · Summarizing One Participant

Once both the questionnaire and task data are cleaned, the next step is to summarize them in a structured, participant-level format. Each participant will eventually become one row in a larger dataset, so it is important to build a consistent structure that captures all the key variables you want to analyze later.

You will write this code in scripts/process_participant.R.
This function will handle a single participant file so that it can later be applied to all participants automatically.

Below is a scaffold that outlines the logic. Read through the structure, then start filling in the details using your own code.

#### process_participant.R -----------------------------------------------------
## Purpose: Given ONE participant .csv, return a one-row summary with:
## subject_id, questionnaire_score, practice mean RT/acc, magnitude mean RT/acc, parity mean RT/acc.

## 0) Input and setup
## Example:
## participant_data <- read.csv(file_path, stringsAsFactors = FALSE)
##
##    Accept a single file path as input.
##    Read the .csv into R and create a subject_id using basename() and sub() to remove ".csv".

## 1) Extract and score the questionnaire
##    Find the row where trialType == "questionnaire".
##    Get the JSON string from the response column.
##    Call score_questionnaire(json_string) to get the participant’s score.

## 2) Split the task data into subsets
## Example:
## practice_data <- subset(participant_data, block == "practice")
##
##    Now create similar subsets for:
##    - the experiment block
##    - magnitude trials within the experiment
##    - parity trials within the experiment

## 3) Filter RTs
## Example:
## practice_filtered <- practice_data[practice_data$rt >= 250 & practice_data$rt <= 900, ]
##
##    Apply the same RT filter (250–900 ms) to each subset.
##    Use clear variable names to track which dataset each one refers to.

## 4) Compute mean RT and accuracy
## Example:
## practice_mean_rt <- mean(practice_filtered$rt, na.rm = TRUE)
## practice_acc <- mean(practice_filtered$correct, na.rm = TRUE)
##
##    Repeat this pattern for magnitude and parity data.
##    Each mean and accuracy should describe one subset of trials.

## 5) Return a one-row data frame with descriptive column names
## Example:
## data.frame(subject_id = subject_id,
##            q_score = questionnaire_score,
##            practice_mean_rt = practice_mean_rt,
##            practice_acc = practice_acc,
##            magnitude_mean_rt = magnitude_mean_rt,
##            magnitude_acc = magnitude_acc,
##            parity_mean_rt = parity_mean_rt,
##            parity_acc = parity_acc)

At this point, your workflow can extract, clean, and summarize data for one participant. Before scaling this up, review the logic behind reverse scoring to make sure the scoring function aligns conceptually with the questionnaire design.

6 · Reverse Scoring Explained

Reverse scoring ensures that all items point in the same conceptual direction.
If a high number on one item means “more engagement” but a high number on another means “less engagement,” the overall score would be meaningless unless the reversed items are corrected.

For a 0–4 scale (jsPsych default):
reversed_value = 4 − original_value

Example:
If a participant chooses 3 (“Agree”) on item 2, which is reverse-scored, the corrected value becomes 4 − 3 = 1.

For a 1–5 scale, the equivalent formula would be:
reversed_value = 6 − original_value

7 · Quality Checks

Before moving on, confirm that:

The questionnaire function returns a numeric score in the expected range.
The filtering correctly removed implausible RTs.
Accuracy values fall between 0 and 1.
You have valid summaries for practice, magnitude, and parity.

You can use functions like summary(), table(), and is.na() to inspect your cleaned data and confirm that the expected ranges and values make sense. For example, summary(practice_filtered$rt) can help verify that all RTs fall between 250 and 900 ms.

Understanding what your data look like at each stage is an essential part of good data management.

8 · Summary

This section focused on turning raw participant data into meaningful summary measures.
You created a questionnaire score from a JSON object using a 0–4 Likert scale, with items 2, 4, and 7 reverse scored.
You also filtered out implausible reaction times and calculated mean RT and accuracy for both the practice and experiment blocks.

The next step will be to combine these processes into a single function that runs automatically across all participant files, saving both intermediate and combined outputs for analysis.