Primary outcomes
The Positive and Negative Affect Schedule
The Positive and Negative Affect Schedule (PANAS) is a widely used scale of emotion and comprises 20 items, 10 measuring positive affect (excited, inspired, etc.) and 10 measuring negative affect (upset, afraid, etc.), on a Likert Scale of 1 (very slightly) to 5 (extremely) [11]. This will be asked with reference to the previous seven days at baseline and 6-week follow-up, the previous 3 days at day 5 and at that precise moment twice daily during the 3-day intervention phase. The PANAS is a reliable and valid measure of depression and anxiety in young adults. The PANAS was chosen for this study as it is relatively stable but also sensitive to mood fluctuations if used with short-term instructions (e.g. now) [11]. The utility of this measure is enhanced by the provision of large-scale normative data in the general UK population [12]. The basic psychometric data for the PANAS was developed from studies in undergraduates, of a similar age range as the participants of this current study, and it is well validated in young adult populations [11].
In previous studies, negative and positive affect have emerged consistently as two dominant and relatively independent dimensions in the structure of affect. The Negative Affect (NA) scale of the PANAS is highly internally consistent, largely uncorrelated, and stable (Cronbach alpha reliabilities for intercorrelations and internal consistency range from .84 to .87) [11]. NA reflects the degree of subjective distress arising from various negative mood states, such as guilt, anger, fear and nervousness. High NA is related to self-reported stress and difficulty coping with negative events. The 10-item Positive Affect (PA) scale is valid and reliable with high internal consistency. Reliabilities range from .86 to .90 [11]. High PA is characterized by energy, concentration and engagement [11].
Psychological distress
The K10 is a well validated self-report measure of psychological distress [13]. The K10 comprises ten questions inquiring about the frequency of depressive and anxiety symptoms over the previous 4 weeks. The K10 has been used as an epidemiological measure to screen for clinically significant depression and anxiety in community samples worldwide. The K10 strongly discriminates between community cases and non-cases of mental disorders identified by a structured clinical interview [14]. In addition, it has been used as an outcome measure in intervention studies [13]. The K10 will be asked with reference to the previous 4 weeks at baseline and 6-week follow-up, and with reference to the previous 3 days at day 5. The K10 is a reliable measure with all items of relevance to young people of this age range [15].
Secondary outcomes
Flourishing
The Flourishing Scale is an 8-item measure of self-perceived success in domains such as self-esteem, relationships, optimism and purpose [16]. It is validated in community samples of young people internationally [17] and provides a single psychological well-being score.
Loneliness
The UCLA Loneliness Scale is a self-report inventory that uses a Likert-type scale to assess subjective feelings of loneliness [18]. There is a validated three-item version [19] to which we have added a direct measure of loneliness as recommended by the Office of National Statistics for measuring loneliness in those aged 16 years and over [20].
Online cognitive tasks
Facial Expression Recognition Task
The Facial Expression Recognition Task provides an objective measure of emotional bias which is associated with vulnerability to depression [21]. Facial expressions of anger, fear, disgust, sadness, surprise, are presented across different intensity levels (0–100%, in 10% steps) and participants are asked to indicate the facial expression that they can see. Two hundred fifty trials are presented split into separate blocks. Accuracy, reaction time and misclassifications are computed in this task. Low mood is associated with an increased tendency to pick up on negative facial expressions compared to positive ones [21]. This testing and analyses will be supported by the infrastructure of the NIHR Oxford Health Biomedical Research Centre
Probabilistic Instrumental Learning Task
The Probabilistic Instrumental Learning Task (PILT), adapted for use online, is based on the Instrumental Learning task described by Pessiglione et al. [22]. We chose this task as depression is characterized by deficits in reward-based decision making. In particular, reward sensitivity is thought to relate to anhedonia, a symptom seen in depression which reflects a deficit in the ability to experience pleasure. We hypothesised that the mechanism of action of cultural experiences on depression might involve an improvement in reward processing and therefore changes in the performance of this task may signify changes in the vulnerability to depression relating to a cultural intervention. This could provide vital clues as to the underlying mechanism by which such interventions might work on a neural level.
The PILT has been described in full elsewhere [23, 24]. In brief, the participant gains or loses points by choosing between two paired images. Participants are required on each trial to choose one of two pairs of symbols. One pair is associated with win outcomes (win 20 pence or no change) and the other with loss outcomes (lose 20 pence or no change). Each symbol in the pair corresponds to reciprocal probabilities (0.7 or 0.3) of the associated outcomes occurring. Participants are instructed to pick the symbol they believe is most likely to win (or least likely to lose), with the aim of maximising their monetary pay off. Feedback on the outcome of each trial is given after a choice is made. Participants complete two runs of the task, each with a new set of four symbols. The total amount won, total amount lost, end total, symbol choice and choice consistency are recorded. The explicit aim is to maximise points.
Trial feedback
At the 6-week follow-up, we will include a research experience survey, based on the NIHR Research Experience checklist (https://www.nihr.ac.uk/documents/optimising-the-participant-in-research-experience-checklist/21378?diaryentryid=60465). This survey includes 8 items asking participants to rate various aspects of their research experience on a Likert scale (1—strongly disagree to 5—strongly agree).
Power analysis and sample size
We aim to recruit over 400 participants. We approximated the statistical power to detect a significant effect of WoB on PANAS scores by assuming an effect size of 0.3 standard deviations of group (WoB vs. Ashmolean Website). With 200 participants in each group, and no pre-intervention differences, a Welch’s t test would have 85% probability of rejecting the null hypothesis under these parameters.
Randomisation and blinding
Participants will be randomized using computer generated random sequencing, in blocks of six stratified by gender with an equal allocation ratio. The allocation sequence is to be configured by a data manager blinded to the allocation group. The participants will be allocated by a researcher blinded to all study data except gender during the entire randomisation and allocation procedure. The participants will be aware of which intervention they have been allocated to. The interventions as well as all measures are online and self-report.
Data management
The Gorilla Experiment Builder (www.gorilla.sc) is a cloud-based research platform that allows researchers to design and administer behavioural (reaction-time) experiments online.
The Gorilla database is encrypted. The investigators own the data that has been collected using Gorilla as well as the data. The investigators can generate and access the anonymized data from Gorilla. Data containing personal identifiable information (e.g. email addresses) are stored in a separate database from the anonymised research database; both hosted by Gorilla. Participants who do not complete measures within a reasonable timeframe from allocation will be timed out of the research database. Completed data will be downloaded and safely stored for statistical analysis on the University of Oxford Study drive.
Analysis plan
A descriptive analysis will be performed for the whole population and each group (WoB and Ashmolean Website). Categorical variables will be described, presenting the numbers and frequency of each. Quantitative variables will be described using usual positional and dispersion parameters.
We will perform an intention to treat analysis to include all participants randomised with any outcome data. Participants will continue to be invited for assessments whether or not they use the allocated intervention. Missing item responses are not possible by design. Missing data, if any, will happen due to participants discontinuing with outcome assessments. However, their data for timepoints prior to that will still be included in the multilevel model, so no additional methodological steps are required regarding missing data.
For NA and PA we will separately regress the subscale means on baseline score (Day 1 measurement), an indicator for Group (Ashmolean website / WoB), and contrast code age (16-17 / 18-24), sex (Male / Female), ethnicity (White / Other), relationship status (In / Not in relationship or other), and current antidepressant use (Yes / No). We will include random intercepts for participants. As there are multiple timepoints at which the group comparison could be made, we will allow the Group effect to vary as a random effect across the measurement timepoints. We will perform a similar analysis for the K10, however as there are fewer measurement timepoints for the K10 compared to the PANAS (2 vs 8) we treat time points as fixed effects. This is the same for secondary outcomes, flourishing and loneliness. The 6-week measures are primarily included to assess whether early differences in the performance of cognitive tasks predict changes in the primary outcomes at 6 weeks.
We will conduct subgroup analyses based on age group (16-17 or 18-24), sex, ethnicity, baseline mental disorder, antidepressant use and previous regular use of online culture (more or less than once a month). We hypothesise that mental disorder at baseline, those not on antidepressants and regular users of culture would be more likely to demonstrate group differences.
Embedded qualitative study
An embedded qualitative study will be conducted in order to ascertain participants experience of the interventions and research process to aid interpretation of the quantitative data and ascertain their views regarding the optimal use of the interventions and further testing. More specifically participants will be asked about the following:
Experience of the intervention
-
Helpfulness to mental health
-
Adverse or unintended consequences
-
Optimal time of use
-
Suggestions for improvement for mental health
Experience of the research
-
How the process of participation in the study was experienced
-
How the testing was experienced
-
Suggestions for improvement for the research process and/or testing
Participants who have consented for further contact will be selected purposively and iteratively. Baseline data will be used to ensure heterogeneity by sociodemographic characteristics, mental health status and previous use of online culture. Participants will be emailed to invite them to consent for a focus group. Focus groups will be conducted separately for each intervention arm at a time convenient to the participants via Zoom. These will be recorded and transcribed for analysis by a multidisciplinary team with expertise in qualitative methodologies. We aim for at least 12 participants to be included in focus groups. When assessing thematic saturation, the sample size will be reviewed [25].
At the 6-week follow-up, there will be free text items regarding what was positive about the research experience and what could be improved. There will also be items relating to the experience of using the allocated intervention and how its use could be optimised for mental health, including two free text options, ‘Please describe any noticeable positive or negative impacts of your allocated cultural experience’ and ‘Are there any ways in which the allocated intervention could be improved for mental health, in your opinion?’
Transcripts and free text responses will be analysed separately. Both transcripts and free text responses will be coded using specialist computer software, NVIVO [26]. Data analysis will take a sequential-explanatory approach [27]. We will combine deductive and inductive methods to investigate whether the theory developed in the preparatory phase holds true, but we will also be open to emerging themes. Analytic meetings will be held with the research team to discuss and optimise validity. Attention will be paid to integrating quantitative and qualitative aspects for evaluating the trial and its interventions [28].
Monitoring
The study will be overseen by a Trial Steering Committee. The Trial Steering Committee will meet regularly and at key stages of the trial, at a minimum at planning, pre-recruitment and prior to completion. The Project Management group will meet at least weekly for the duration of the trial. Any data-related issues, should they arise will be discussed within the Project Management group and with the Trial Steering Committee at the next meeting, or meetings will be expedited if deemed necessary. Access to study data will be provided to authorised representatives from the Sponsor for monitoring and audit purposes if required.
The Trial Steering Committee consists of the Head of Department and researchers from the University of Oxford Department of Psychiatry, a Consultant Psychiatrist, two staff members from Oxford University Gardens, Libraries and Museums, three Public and Patient involvement (PPI) members, researchers from the Oxford Internet Institute (including the Director of Research) and three students and members of staff from the target age range. The Trial Steering Committee decided that a separate Data Monitoring Committee would not be necessary based on the short time frame of the intervention phase and low risk nature of the trial.
Both interventions and trial procedures relevant to the first 5 days of the study have already been trialled on eleven volunteers and detailed feedback taken and discussed in detail by the Trial Steering Committee. This is a low-risk study, and we do not anticipate any reasons for unblinding; therefore, study-specific interim analyses, stopping or unblinding procedures have not been put into place. Should the need arise, the relevant University of Oxford standard operating procedures will be followed.
