Last updated: 2022-11-09

Checks: 7 0

Knit directory: SCHEDULING/

This reproducible R Markdown analysis was created with workflowr (version 1.7.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Environment: empty

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

Seed: set.seed(20221107)

The command set.seed(20221107) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Session information: recorded

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Cache: none

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

File paths: relative

Great job! Using relative paths to the files within your workflowr project makes it easier to run your code on other machines.

Repository version: 67e1aac

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 67e1aac. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:


Ignored files:
    Ignored:    .Rproj.user/
    Ignored:    analysis/patch_selection.png
    Ignored:    analysis/patch_selection_8.png
    Ignored:    analysis/patch_selection_avg.png
    Ignored:    analysis/site_libs/

Untracked files:
    Untracked:  analysis/Notes.txt
    Untracked:  analysis/archive/
    Untracked:  analysis/fd_pl.rds
    Untracked:  analysis/fu_pl.rds
    Untracked:  analysis/prereg/
    Untracked:  analysis/reward rate analysis.docx
    Untracked:  analysis/rewardRate.jpg
    Untracked:  analysis/toAnalyse/
    Untracked:  analysis/wflow_code_string.txt
    Untracked:  archive/
    Untracked:  data/archive/
    Untracked:  data/create_database.sql
    Untracked:  data/dataToAnalyse/
    Untracked:  data/exp6a_typing_exponential.xlsx
    Untracked:  data/exp6b_typing_linear.xlsx
    Untracked:  data/rawdata_incEmails/
    Untracked:  data/sona data/
    Untracked:  data/summaryFiles/
    Untracked:  models/
    Untracked:  old Notes on analysis.txt
    Untracked:  presentations/
    Untracked:  references/
    Untracked:  spatial_pdist.Rdata

Unstaged changes:
    Modified:   data/README.md

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/analysis_exp7c_rdk_reward_instructions.Rmd) and HTML (docs/analysis_exp7c_rdk_reward_instructions.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File	Version	Author	Date	Message
Rmd	67e1aac	knowlabUnimelb	2022-11-09	Publish data and analysis files

Sandra Takchi¹, Ami Eidels², and Daniel R. Little¹ ¹ The University of Melbourne, ² The University of Newcastle

Method

Participants

53 participants were recruited through the Melbourne School of Psychological Sciences Research Experience Program, and they were reimbursed by earning one credit for each hour of participation. 38 of the participants were females and 9 were males, and their mean age was 19.2 with a standard deviation of 2.4. Participants who did not complete the whole experiment or were not able to learn the task were excluded from the experiment. 44 participants remained in the analysis after these exclusion.

Design

Participants completed a series of trials of selecting and completing RDK tasks. They were instructed to make judgment about the direction of the dots in the RDK tasks, where they attempted to complete as many subtasks as possible before the deadline. In addition, a wordle game was introduced, and participants gained clues that varied by value depending on the order of subtask difficulty they followed. The reward with the highest value was gained when they started with the most difficult task and the one with the least value was gained when they started with the easiest task.

Data Cleaning

Subjects completed the experiment by clicking a link with the uniquely generated id code. Subjects were able to use the link multiple times; further, subjects were able to exit the experiment at any time. Consequently, the datafile contains partially completed data for some subjects which needed to be identified and removed.

We identified non-learners as participants who had less than 40% accuracy on the easiest task. These participants were removed.

Data Analysis

We first summarized performance by answering the following questions:

Task completions

How many tasks are completed on average?

We assumed that by adding a deadline, participants will complete less tasks then when it is untimed. Hence, we inspected this assumption by calculating the average number of task completions for the timed and untimed phases.

Average number of correctly completed tasks in each condition
phase	mean
untimed	3.945026
deadline	3.344948

As expected, in the untimed phase, average correct completed tasks were higher than the phase with the deadline. The deadline reduced the number of tasks completed.

RDK performance

We next analysed performance on the RDK discriminations. We then asked:

What was the average completion time and accuracy of the easy, medium, hard, and very hard tasks?

We assumed that participants will spend the most time and have a lower accuracy on the most difficult task compared to the easiest one. To check this effect of difficulty, we calculated the average completion time and accuracy of each difficulty.

On average, accuracy decreased as the task difficulty increased, and response time increased as difficulty increased.

We further broke down RTs by condition, deadline, and difficulty.

Reward Rate

Reward rate is the reward amount of each task divided by the time needed to complete it. The optimal order is determined by ordering the tasks according to the tasks which produces the highest value reward per unit time.

ANOVA Table (type III tests)

$ANOVA Effect DFn DFd F p p<.05 ges 1 phase 1 36 12.199 1.00e-03 * 0.055 2 difficulty 3 108 22.185 2.88e-11 * 0.166 3 phase:difficulty 3 108 6.445 4.69e-04 * 0.040

$Mauchly's Test for Sphericity Effect W p p<.05 1 difficulty 0.006 2.87e-36 2 phase:difficulty 0.009 3.63e-33

$Sphericity Corrections Effect GGe DF[GG] p[GG] p[GG]<.05 HFe DF[HF] 1 difficulty 0.398 1.19, 42.95 9.21e-06 * 0.404 1.21, 43.59 2 phase:difficulty 0.438 1.31, 47.33 9.00e-03 * 0.448 1.34, 48.41 p[HF] p[HF]<.05 1 8.12e-06 2 8.00e-03

Pairwise comparisons using t tests with pooled SD

data: rrdata$rewardRate and paste(rrdata$difficulty, rrdata$phase)

             easy deadline easy untimed hard deadline hard untimed

easy untimed 1.00000 - - -
hard deadline 1.1e-05 6.0e-06 - -
hard untimed 0.77638 0.57922 0.09148 -
medium deadline 1.00000 1.00000 0.01116 1.00000
medium untimed 1.00000 1.00000 0.00073 1.00000
v. hard deadline 1.9e-10 9.3e-11 1.00000 4.3e-05
v. hard untimed 0.26832 0.19225 0.26928 1.00000
medium deadline medium untimed v. hard deadline easy untimed - - -
hard deadline - - -
hard untimed - - -
medium deadline - - -
medium untimed 1.00000 - -
v. hard deadline 1.9e-06 5.0e-08 -
v. hard untimed 1.00000 1.00000 0.00021

P value adjustment method: bonferroni

Mean accuracy, RT, cumulative RT, and Reward Rate for each difficulty and each phase
phase	difficulty	n	Mean.Correct	SE.Correct	Mean.RT	SE.RT	Mean.crt	SE.crt	Mean.RR	SE.RR
untimed	easy	585	0.94	0.01	1310.93	75.30	1393.50	85.05	0.1045477	0.0022227
untimed	medium	475	0.92	0.01	934.39	32.53	1018.43	39.00	0.8623589	0.0190445
untimed	hard	412	0.79	0.02	1118.91	41.46	1417.29	69.38	1.8465185	0.0701345
untimed	v. hard	405	0.64	0.02	1448.05	68.78	2241.04	118.53	2.1299166	0.1004513
deadline	easy	1490	0.94	0.01	580.54	9.21	613.80	10.60	0.1899873	0.0027108
deadline	medium	1150	0.90	0.01	517.97	7.09	570.27	8.80	1.3994441	0.0243685
deadline	hard	1134	0.80	0.01	560.57	8.63	669.43	13.30	4.0816259	0.1128912
deadline	v. hard	1015	0.63	0.02	646.52	12.46	948.14	25.51	5.5428413	0.2549319

Optimality in each condition

What is the proportion of easy, medium, hard, and very hard tasks selected first, second, third or fourth?

To assess participants’ optimality, we looked into the order in which they adopted, visualized in heat maps that portray the proportion of selecting tasks of particular difficulties.

Do the marginal distributions differ from uniformity?

We tested whether the marginal distributions were different from uniformally random selection using the fact that the mean rank is distributed according to a $\chi^2$ distribution with the following test-statistic: \[\chi^2 = \frac{12N}{k(k+1)}\sum_{j=1}^k \left(m_j - \frac{k+1}{2} \right)^2\] see (Marden, 1995).

Chi2 test of uniformity
phase	chi2	df	p
untimed	319.02	3	0
deadline	84.28	3	0

A chi-square test of uniformity was significant, indicating that participants did not choose tasks of different difficulty by random.

We compared the location conditions and phases using chi-2 analysis.

Pearson’s chi-squared test
	Comparison	chi2	df	p
X-squared	Untimed vs Deadline	162.78	15	0

A Pearson’s chi-square test indicated that there was a significant difference between the untimed and deadline phases in the order participants adopted.

K-S test was significant, meaning that the probability distributions between timed and untimed phases are not the same.

How optimal were responses?

Participants were not optimal, since they followed an easy to hard order even when the hardest task produced the highest value reward.

Comparison against Easy to Hard order

How consistent were responses with an easy to hard ordering?

Wordle-Clue Score Analysis

How motivated are participants to get the Wordle-Clue guess correct?

Wordle guesses are scored out of 10. A match-in-place is scored 2; a match-out-of-place is scored 1. The final score for each trial is the sum across all letters.

Average Wordle Scores (Max Score = 10)
phase	score
untimed	6.058537
deadline	5.402121

        Df Sum Sq Mean Sq F value Pr(>F)

phase 1 8.83 8.834 4.119 0.0457 * Residuals 80 171.57 2.145
— Signif. codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ’ ’ 1

What is the correlation between the Wordle-Clue score and the distance from the optimal schedule? From the easy-hard schedule?

This test indicates whether participants who are more motivated to score highly on the wordle test are also more likely to selection optimal schedules. This would be indicated by a significant correlation (high wordle score coupled with a low average distance). We also test the same wordle score compared to the easy-hard schedule.

[1] -0.119347

Pearson's product-moment correlation

data: corrDataSet$wordleScore and corrDataSet$d t = -1.0616, df = 78, p-value = 0.2917 alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: -0.3303999 0.1030728 sample estimates: cor -0.119347

[1] -0.01394418

Pearson's product-moment correlation

data: corrDataSet$wordleScore[corrDataSet$phase == “untimed”] and corrDataSet$d[corrDataSet$phase == “untimed”] t = -0.085966, df = 38, p-value = 0.9319 alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: -0.3240460 0.2988634 sample estimates: cor -0.01394418

[1] -0.3496954

Pearson's product-moment correlation

data: corrDataSet$wordleScore[corrDataSet$phase == “deadline”] and corrDataSet$d[corrDataSet$phase == “deadline”] t = -2.3009, df = 38, p-value = 0.02697 alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: -0.59625290 -0.04285431 sample estimates: cor -0.3496954

[1] 0.119347

Pearson's product-moment correlation

data: corrDataSet$wordleScore and corrDataSet$eh_d t = 1.0616, df = 78, p-value = 0.2917 alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: -0.1030728 0.3303999 sample estimates: cor 0.119347

sessionInfo()

R version 4.1.3 (2022-03-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=English_Australia.1252  LC_CTYPE=English_Australia.1252   
[3] LC_MONETARY=English_Australia.1252 LC_NUMERIC=C                      
[5] LC_TIME=English_Australia.1252    

attached base packages:
[1] stats4    grid      stats     graphics  grDevices utils     datasets 
[8] methods   base     

other attached packages:
 [1] pmr_1.2.5.1       jpeg_0.1-9        rstatix_0.7.0     lme4_1.1-29      
 [5] Matrix_1.4-0      png_0.1-7         reshape2_1.4.4    knitr_1.38       
 [9] hrbrthemes_0.8.0  english_1.2-6     gtools_3.9.2      DescTools_0.99.45
[13] forcats_0.5.1     stringr_1.4.0     dplyr_1.0.8       purrr_0.3.4      
[17] readr_2.1.2       tidyr_1.2.0       tibble_3.1.6      ggplot2_3.3.5    
[21] tidyverse_1.3.1   rmarkdown_2.13    workflowr_1.7.0  

loaded via a namespace (and not attached):
 [1] minqa_1.2.4       colorspace_2.0-3  ellipsis_0.3.2    class_7.3-20     
 [5] rprojroot_2.0.3   fs_1.5.2          gld_2.6.5         rstudioapi_0.13  
 [9] proxy_0.4-27      farver_2.1.0      fansi_1.0.3       mvtnorm_1.1-3    
[13] lubridate_1.8.0   xml2_1.3.3        splines_4.1.3     extrafont_0.18   
[17] rootSolve_1.8.2.3 jsonlite_1.8.0    nloptr_2.0.0      broom_0.8.0      
[21] Rttf2pt1_1.3.10   dbplyr_2.1.1      compiler_4.1.3    httr_1.4.2       
[25] backports_1.4.1   assertthat_0.2.1  fastmap_1.1.0     cli_3.2.0        
[29] later_1.3.0       htmltools_0.5.2   tools_4.1.3       gtable_0.3.0     
[33] glue_1.6.2        lmom_2.9          Rcpp_1.0.8.3      carData_3.0-5    
[37] cellranger_1.1.0  jquerylib_0.1.4   vctrs_0.4.1       nlme_3.1-155     
[41] extrafontdb_1.0   xfun_0.30         ps_1.6.0          rvest_1.0.2      
[45] lifecycle_1.0.1   getPass_0.2-2     MASS_7.3-55       scales_1.2.0     
[49] hms_1.1.1         promises_1.2.0.1  expm_0.999-6      yaml_2.3.5       
[53] Exact_3.1         gdtools_0.2.4     sass_0.4.1        stringi_1.7.6    
[57] highr_0.9         e1071_1.7-11      boot_1.3-28       rlang_1.0.2      
[61] pkgconfig_2.0.3   systemfonts_1.0.4 evaluate_0.15     lattice_0.20-45  
[65] labeling_0.4.2    processx_3.5.3    tidyselect_1.1.2  plyr_1.8.7       
[69] magrittr_2.0.3    R6_2.5.1          generics_0.1.2    DBI_1.1.2        
[73] pillar_1.7.0      haven_2.5.0       whisker_0.4       withr_2.5.0      
[77] abind_1.4-5       modelr_0.1.8      crayon_1.5.1      car_3.0-12       
[81] utf8_1.2.2        tzdb_0.3.0        readxl_1.4.0      data.table_1.14.2
[85] callr_3.7.0       git2r_0.30.1      reprex_2.0.1      digest_0.6.29    
[89] httpuv_1.6.5      munsell_0.5.0     bslib_0.3.1

Experiment 7c: Labelled RDK - Wordle Reward - Reward instructions

knowlabUnimelb

2022-08-07