Stats analysis for Mat

Intro
Questions
Load the data
Matt’s direction for statistical analysis
Conclusions
Session information

Source code: https://github.com/markziemann/survey_stats_mat

Intro

A survey based study into student preferences for a polling tool in revision lessons.

Questions

Below are the survey questions given to students.

Q1. “For Revision Class 1 (on Tuesday 6th August), did you:”

face-to-face
recording
livestream
did not attend

Q2. “If you attended Revision Class 1, did you answer any questions aloud in front of the class?”

Yes, 1
Yes >1
No

Q3. “How comfortable did you feel to answer questions in Revision Class 1?”

Extremely comfortable
Neutral
Extremely comfortable

Q4. “How would you rate the usefulness of Revision Class 1 for understanding the topics covered?”

Extremely useful
Very useful
Moderately useful
Slightly useful
Not useful at all

Q5. “How would you rate your overall satisfaction with Revision Class 1?”

Very Satisfied
Satisfied
Somewhat Satisfied
Neutral
Somewhat Dissatisfied
Dissatisfied
Very Dissatisfied

Q6. “Any comments about Revision Class 1 and answering questions aloud during the class?”

Qualitative answer in text box.

Q7. “For Revision Class 2 (Monday 26th August, using the Socrative App), did you:”

Face-to-face
Recording
Livestream
Did not attend

Q8. “If you attended Revision Class 2, did you use the Socrative App to answer any questions?”

Yes, 1
Yes >1
No

Q9. “How comfortable did you feel to answer questions in Revision Class 2 using the Socrative App?”

Extremely comfortable
Neutral
Extremely comfortable

Q10. “How would you rate the usefulness of Revision Class 2 for understanding the topics covered?”

Extremely useful
Very useful
Moderately useful
Slightly useful
Not useful at all

Q11. “How would you rate your overall satisfaction with Revision Class 2?”

Very Satisfied
Satisfied
Somewhat Satisfied
Neutral
Somewhat Dissatisfied
Dissatisfied
Very Dissatisfied

Q12. “Any comments about Revision Class 2 and using the Socrative App?”

Qualitative answer in text box.

Q13. “Overall, did you prefer Revision Class 1 (traditional method of answering questions aloud during the class), or Revision class 2 (using the Socrative App to answer questions)?”

Class 1 with traditional method
Class 2 with Socrative

Q14. “For future revision classes, which method would you prefer to be used?”

Traditional method
Socrative method

Load the data

x <- read.csv("Revision Class Survey SLE346 T2 2019_September 12, 2019_11.26_for MZ fmt.csv")
head(x)

##          StartDate          EndDate     Status      IPAddress Progress
## 1  27/08/2019 9:53  27/08/2019 9:55 IP Address 128.184.189.40      100
## 2 27/08/2019 10:22 27/08/2019 10:24 IP Address 128.184.189.40      100
## 3 27/08/2019 10:24 27/08/2019 10:25 IP Address 128.184.189.40      100
## 4 27/08/2019 10:27 27/08/2019 10:28 IP Address 128.184.189.40      100
## 5 27/08/2019 10:28 27/08/2019 10:29 IP Address 128.184.189.40      100
## 6 27/08/2019 10:29 27/08/2019 10:30 IP Address 128.184.189.40      100
##   Duration..in.seconds. Finished     RecordedDate        ResponseId
## 1                    96     TRUE  27/08/2019 9:55 R_10pmhlMdMQkd1RC
## 2                    66     TRUE 27/08/2019 10:24 R_3EHk5acI62QdeAa
## 3                    82     TRUE 27/08/2019 10:25 R_9Yse4r7oMuWjhMR
## 4                    55     TRUE 27/08/2019 10:28 R_3psTOaN2YhV9W2X
## 5                    49     TRUE 27/08/2019 10:29 R_3EFrLhU2bjIytEf
## 6                    54     TRUE 27/08/2019 10:30 R_1rB1ToP69zj3y3O
##   RecipientLastName RecipientFirstName RecipientEmail ExternalReference
## 1                NA                 NA             NA                NA
## 2                NA                 NA             NA                NA
## 3                NA                 NA             NA                NA
## 4                NA                 NA             NA                NA
## 5                NA                 NA             NA                NA
## 6                NA                 NA             NA                NA
##   LocationLatitude LocationLongitude DistributionChannel UserLanguage
## 1         -38.2097          144.3022           anonymous           EN
## 2         -38.2097          144.3022           anonymous           EN
## 3         -38.2097          144.3022           anonymous           EN
## 4         -38.2097          144.3022           anonymous           EN
## 5         -38.2097          144.3022           anonymous           EN
## 6         -38.2097          144.3022           anonymous           EN
##                                 Q1                           Q2
## 1   Attend the class face-to-face?                           No
## 2   Attend the class face-to-face?                           No
## 3   Attend the class face-to-face? Yes (more than one question)
## 4   Attend the class face-to-face?                           No
## 5   Attend the class face-to-face?                           No
## 6 Watch a livestream of the class?                           No
##                                      Q3               Q4                 Q5
## 1 Neither comfortable nor uncomfortable Extremely useful     Very Satisfied
## 2                Moderately comfortable      Very useful          Satisfied
## 3                Moderately comfortable      Very useful          Satisfied
## 4                 Extremely comfortable      Very useful          Satisfied
## 5                Moderately comfortable      Very useful Somewhat Satisfied
## 6                                            Very useful          Satisfied
##                                                                       Q6
## 1                                                                       
## 2                                                                       
## 3 Might have been good to have a quick topic revision then the questions
## 4                                                                       
## 5                                                                       
## 6                                                                       
##                               Q7                                       Q8
## 1 Attend the class face-to-face? Yes (all or almost all of the questions)
## 2 Attend the class face-to-face? Yes (all or almost all of the questions)
## 3 Attend the class face-to-face? Yes (all or almost all of the questions)
## 4 Attend the class face-to-face?    Yes (but only a few of the questions)
## 5 Attend the class face-to-face?                                       No
## 6 Attend the class face-to-face?    Yes (but only a few of the questions)
##                       Q9               Q10            Q11       Q12
## 1  Extremely comfortable  Extremely useful Very Satisfied          
## 2  Extremely comfortable Moderately useful      Satisfied          
## 3  Extremely comfortable       Very useful      Satisfied As before
## 4 Moderately comfortable       Very useful      Satisfied          
## 5 Moderately comfortable       Very useful      Satisfied          
## 6   Slightly comfortable  Extremely useful Very Satisfied          
##                            Q13                                Q14 Q12...Topics
## 1    Class 2 (using Socrative)   Socrative Method (Class 2 style)             
## 2 Class 1 (traditional method)   Socrative Method (Class 2 style)             
## 3    Class 2 (using Socrative)   Socrative Method (Class 2 style)      Unknown
## 4 Class 1 (traditional method) Traditional Method (Class 1 style)             
## 5    Class 2 (using Socrative)   Socrative Method (Class 2 style)             
## 6    Class 2 (using Socrative)   Socrative Method (Class 2 style)             
##   Q12...Parent.Topics Q6...Topics Q6...Parent.Topics
## 1                  NA                             NA
## 2                  NA                             NA
## 3                  NA     Unknown                 NA
## 4                  NA                             NA
## 5                  NA                             NA
## 6                  NA                             NA

str(x)

## 'data.frame':    18 obs. of  35 variables:
##  $ StartDate            : chr  "27/08/2019 9:53" "27/08/2019 10:22" "27/08/2019 10:24" "27/08/2019 10:27" ...
##  $ EndDate              : chr  "27/08/2019 9:55" "27/08/2019 10:24" "27/08/2019 10:25" "27/08/2019 10:28" ...
##  $ Status               : chr  "IP Address" "IP Address" "IP Address" "IP Address" ...
##  $ IPAddress            : chr  "128.184.189.40" "128.184.189.40" "128.184.189.40" "128.184.189.40" ...
##  $ Progress             : int  100 100 100 100 100 100 100 100 100 100 ...
##  $ Duration..in.seconds.: int  96 66 82 55 49 54 66 110 49 54 ...
##  $ Finished             : logi  TRUE TRUE TRUE TRUE TRUE TRUE ...
##  $ RecordedDate         : chr  "27/08/2019 9:55" "27/08/2019 10:24" "27/08/2019 10:25" "27/08/2019 10:28" ...
##  $ ResponseId           : chr  "R_10pmhlMdMQkd1RC" "R_3EHk5acI62QdeAa" "R_9Yse4r7oMuWjhMR" "R_3psTOaN2YhV9W2X" ...
##  $ RecipientLastName    : logi  NA NA NA NA NA NA ...
##  $ RecipientFirstName   : logi  NA NA NA NA NA NA ...
##  $ RecipientEmail       : logi  NA NA NA NA NA NA ...
##  $ ExternalReference    : logi  NA NA NA NA NA NA ...
##  $ LocationLatitude     : num  -38.2 -38.2 -38.2 -38.2 -38.2 ...
##  $ LocationLongitude    : num  144 144 144 144 144 ...
##  $ DistributionChannel  : chr  "anonymous" "anonymous" "anonymous" "anonymous" ...
##  $ UserLanguage         : chr  "EN" "EN" "EN" "EN" ...
##  $ Q1                   : chr  "Attend the class face-to-face?" "Attend the class face-to-face?" "Attend the class face-to-face?" "Attend the class face-to-face?" ...
##  $ Q2                   : chr  "No" "No" "Yes (more than one question)" "No" ...
##  $ Q3                   : chr  "Neither comfortable nor uncomfortable" "Moderately comfortable" "Moderately comfortable" "Extremely comfortable" ...
##  $ Q4                   : chr  "Extremely useful" "Very useful" "Very useful" "Very useful" ...
##  $ Q5                   : chr  "Very Satisfied" "Satisfied" "Satisfied" "Satisfied" ...
##  $ Q6                   : chr  "" "" "Might have been good to have a quick topic revision then the questions" "" ...
##  $ Q7                   : chr  "Attend the class face-to-face?" "Attend the class face-to-face?" "Attend the class face-to-face?" "Attend the class face-to-face?" ...
##  $ Q8                   : chr  "Yes (all or almost all of the questions)" "Yes (all or almost all of the questions)" "Yes (all or almost all of the questions)" "Yes (but only a few of the questions)" ...
##  $ Q9                   : chr  "Extremely comfortable" "Extremely comfortable" "Extremely comfortable" "Moderately comfortable" ...
##  $ Q10                  : chr  "Extremely useful" "Moderately useful" "Very useful" "Very useful" ...
##  $ Q11                  : chr  "Very Satisfied" "Satisfied" "Satisfied" "Satisfied" ...
##  $ Q12                  : chr  "" "" "As before" "" ...
##  $ Q13                  : chr  "Class 2 (using Socrative)" "Class 1 (traditional method)" "Class 2 (using Socrative)" "Class 1 (traditional method)" ...
##  $ Q14                  : chr  "Socrative Method (Class 2 style)" "Socrative Method (Class 2 style)" "Socrative Method (Class 2 style)" "Traditional Method (Class 1 style)" ...
##  $ Q12...Topics         : chr  "" "" "Unknown" "" ...
##  $ Q12...Parent.Topics  : logi  NA NA NA NA NA NA ...
##  $ Q6...Topics          : chr  "" "" "Unknown" "" ...
##  $ Q6...Parent.Topics   : logi  NA NA NA NA NA NA ...

Matt’s direction for statistical analysis

For the comparison, I am mainly interested in the responses comparing Class 1 (verbal answers) with Class 2 (Socrative app).

I am not so worried about breaking the students down into in-person, watched recording or watched livestream; it could be interesting but I don’t think the numbers are anywhere big enough to get any useful data out of it.

So in my original analyses I am mostly interested in the following questions:

Was there a difference in participation between traditional and socrative classes?

The first approach to answer will be to use the three levels of participation and test with a Wilcox test and generate the confidence intervals with a t-test.

# participation for class 1
part <- x$Q2
part <- part[which(part!="")]
part <- gsub('Yes \\(more than one question\\)',"2",part)
part <- gsub("No","0",part)
part1 <- as.numeric(part)
part1

##  [1] 0 0 2 0 0 0 0 0 0 0 0 2 0 2 0 0

# participation for class 2
part <- x$Q8
part <- part[which(part!="")]
part <- gsub('Yes \\(but only a few of the questions\\)',"1",part)
part <- gsub('Yes \\(all or almost all of the questions\\)',"2",part)
part <- gsub("No","0",part)
part2 <- as.numeric(part)
part2

##  [1] 2 2 2 1 0 1 1 2 2 2 0 2 1 2 1 2

par(mfrow=c(2,1), cex=0.8)
hist(part1)
hist(part2)

wilcox.test(part1,part2)

## Warning in wilcox.test.default(part1, part2): cannot compute exact p-value with
## ties

## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  part1 and part2
## W = 47.5, p-value = 0.001012
## alternative hypothesis: true location shift is not equal to 0

t.test(part1,part2)

## 
##  Welch Two Sample t-test
## 
## data:  part1 and part2
## t = -3.9138, df = 29.688, p-value = 0.0004902
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -1.6171673 -0.5078327
## sample estimates:
## mean of x mean of y 
##    0.3750    1.4375

According to the t-test the participation was significantly higher in class 2.

An alternative approach would be to perform binary classify of students as non-participants or participants and then run a Fisher exact test.

part1

##  [1] 0 0 2 0 0 0 0 0 0 0 0 2 0 2 0 0

class1_nonparticipants = length(which(part1==0))
class1_participants = length(which(part1!=0))
class2_nonparticipants = length(which(part2==0))
class2_participants = length(which(part2!=0))
row1 <- c(class1_nonparticipants,class1_participants)
row2 <- c(class2_nonparticipants,class2_participants)
y <- rbind(row1,row2)
rownames(y) <- c("class1","class2")
colnames(y) <- c("non participation","participation")
y

##        non participation participation
## class1                13             3
## class2                 2            14

fisher.test(y)

## 
##  Fisher's Exact Test for Count Data
## 
## data:  y
## p-value = 0.0002444
## alternative hypothesis: true odds ratio is not equal to 1
## 95 percent confidence interval:
##    3.443008 356.526674
## sample estimates:
## odds ratio 
##   25.74352

# participation rate class 1 (%)
p1 = y[1,2]/sum(y[1,])*100
p1

## [1] 18.75

# participation rate class 2 (%)
p2 = y[2,2]/sum(y[2,]) *100
p2

## [1] 87.5

# increase in participation (%)
p2/p1 * 100

## [1] 466.6667

Again the p-value is very small indicating a significantly higher participation rate in the Socrative class.

Were students more comfortable to participate in answering questions in traditional or Socrative classes?

I have inverted the values to make it a measure of comfort. That means bigger numbers are “good”. This can be used in Wilcox test or t-test on ranks. Scale: 1 (extremely uncomf) to 7 (extremely comf).

# class 1 comfort
x$Q3

##  [1] "Neither comfortable nor uncomfortable"
##  [2] "Moderately comfortable"               
##  [3] "Moderately comfortable"               
##  [4] "Extremely comfortable"                
##  [5] "Moderately comfortable"               
##  [6] ""                                     
##  [7] "Neither comfortable nor uncomfortable"
##  [8] "Moderately comfortable"               
##  [9] "Moderately comfortable"               
## [10] "Extremely comfortable"                
## [11] "Slightly uncomfortable"               
## [12] "Extremely comfortable"                
## [13] "Moderately comfortable"               
## [14] "Slightly comfortable"                 
## [15] "Moderately comfortable"               
## [16] "Moderately comfortable"               
## [17] "Moderately uncomfortable"             
## [18] "Moderately uncomfortable"

comf <- x$Q3
comf <- comf[which(comf!="")]
comf <- comf[which(!is.na(comf))]
comf <- gsub("Extremely comfortable","7",comf)
comf <- gsub("Moderately comfortable","6",comf)
comf <- gsub("Slightly comfortable","5",comf)
comf <- gsub("Neither comfortable nor uncomfortable","4",comf)
comf <- gsub("Slightly uncomfortable","3",comf)
comf <- gsub("Moderately uncomfortable","2",comf)
comf <- gsub("Extremely uncomfortable","1",comf)
comf1 <- as.numeric(comf)
comf1

##  [1] 4 6 6 7 6 4 6 6 7 3 7 6 5 6 6 2 2

mean(comf1)

## [1] 5.235294

# class 2 comfort
x$Q9

##  [1] "Extremely comfortable"                
##  [2] "Extremely comfortable"                
##  [3] "Extremely comfortable"                
##  [4] "Moderately comfortable"               
##  [5] "Moderately comfortable"               
##  [6] "Slightly comfortable"                 
##  [7] "Moderately comfortable"               
##  [8] "Extremely comfortable"                
##  [9] "Extremely comfortable"                
## [10] "Extremely comfortable"                
## [11] "Extremely comfortable"                
## [12] "Neither comfortable nor uncomfortable"
## [13] "Extremely comfortable"                
## [14] "Extremely comfortable"                
## [15] "Extremely comfortable"                
## [16] "Extremely comfortable"                
## [17] "Moderately comfortable"               
## [18] "Moderately comfortable"

comf <- x$Q9
comf <- comf[which(comf!="")]
comf <- comf[which(!is.na(comf))]
comf <- gsub("Extremely comfortable","7",comf)
comf <- gsub("Moderately comfortable","6",comf)
comf <- gsub("Slightly comfortable","5",comf)
comf <- gsub("Neither comfortable nor uncomfortable","4",comf)
comf <- gsub("Slightly uncomfortable","3",comf)
comf <- gsub("Moderately uncomfortable","2",comf)
comf <- gsub("Extremely uncomfortable","1",comf)
comf2 <- as.numeric(comf)
comf2

##  [1] 7 7 7 6 6 5 6 7 7 7 7 4 7 7 7 7 6 6

mean(comf2)

## [1] 6.444444

par(mfrow=c(2,1), cex=0.8)
hist(comf1)
hist(comf2)

wilcox.test(comf1,comf2)

## Warning in wilcox.test.default(comf1, comf2): cannot compute exact p-value with
## ties

## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  comf1 and comf2
## W = 76, p-value = 0.007259
## alternative hypothesis: true location shift is not equal to 0

t.test(comf1,comf2)

## 
##  Welch Two Sample t-test
## 
## data:  comf1 and comf2
## t = -2.7107, df = 23.798, p-value = 0.01226
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -2.1301928 -0.2881078
## sample estimates:
## mean of x mean of y 
##  5.235294  6.444444

This result indicates that participants were significantly more comfortable to participate with Socrative as compared to traditional/conventional method.

Did students perceive traditional or Socrative classes to be more useful?

Same approach as above but the scale goes from 1 (not useful at all) to 5 (extremely useful).

# class 1 utility
x$Q4

##  [1] "Extremely useful" "Very useful"      "Very useful"      "Very useful"     
##  [5] "Very useful"      "Very useful"      "Very useful"      "Extremely useful"
##  [9] "Very useful"      "Extremely useful" "Extremely useful" "Extremely useful"
## [13] "Extremely useful" "Very useful"      "Very useful"      "Extremely useful"
## [17] "Very useful"      "Very useful"

us <- x$Q4
us <- us[which(us!="")]
us <- us[which(!is.na(us))]
us <- gsub("Extremely useful","5",us)
us <- gsub("Very useful","4",us)
us <- gsub("Moderately useful","3",us)
us <- gsub("Slightly useful","2",us)
us <- gsub("Not at all useful","1",us)
us1 <- as.numeric(us)
us1

##  [1] 5 4 4 4 4 4 4 5 4 5 5 5 5 4 4 5 4 4

mean(us1)

## [1] 4.388889

# class 2 utility
x$Q10

##  [1] "Extremely useful"  "Moderately useful" "Very useful"      
##  [4] "Very useful"       "Very useful"       "Extremely useful" 
##  [7] "Very useful"       "Very useful"       "Extremely useful" 
## [10] "Extremely useful"  "Extremely useful"  "Extremely useful" 
## [13] "Extremely useful"  "Very useful"       "Extremely useful" 
## [16] "Extremely useful"  "Extremely useful"  "Extremely useful"

us <- x$Q10
us <- us[which(us!="")]
us <- us[which(!is.na(us))]
us <- gsub("Extremely useful","5",us)
us <- gsub("Very useful","4",us)
us <- gsub("Moderately useful","3",us)
us <- gsub("Slightly useful","2",us)
us <- gsub("Not at all useful","1",us)
us2 <- as.numeric(us)
us2

##  [1] 5 3 4 4 4 5 4 4 5 5 5 5 5 4 5 5 5 5

mean(us2)

## [1] 4.555556

par(mfrow=c(2,1), cex=0.8)
hist(us1, xlim = c(1,5))
hist(us2, xlim = c(1,5))

wilcox.test(us1,us2)

## Warning in wilcox.test.default(us1, us2): cannot compute exact p-value with ties

## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  us1 and us2
## W = 131.5, p-value = 0.2795
## alternative hypothesis: true location shift is not equal to 0

t.test(us1,us2)

## 
##  Welch Two Sample t-test
## 
## data:  us1 and us2
## t = -0.89036, df = 32.666, p-value = 0.3798
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.5476547  0.2143213
## sample estimates:
## mean of x mean of y 
##  4.388889  4.555556

Students ranked Socrative classes higher but there was no significant difference.

Were students more satisfied with traditional or Socrative classes?

Satisaction was on a scale of 1 (extremely dissatisfied) to 7 (extremely dissatisfied).

# class 1 satisfaction
x$Q5

##  [1] "Very Satisfied"     "Satisfied"          "Satisfied"         
##  [4] "Satisfied"          "Somewhat Satisfied" "Satisfied"         
##  [7] "Satisfied"          "Satisfied"          "Very Satisfied"    
## [10] "Very Satisfied"     "Very Satisfied"     "Very Satisfied"    
## [13] "Very Satisfied"     "Satisfied"          "Very Satisfied"    
## [16] "Satisfied"          "Satisfied"          "Satisfied"

sat <- x$Q5
sat <- sat[which(sat!="")]
sat <- sat[which(!is.na(sat))]
sat <- gsub("Very Satisfied","7",sat)
sat <- gsub("Somewhat Satisfied","5",sat)
sat <- gsub("Neutral","4",sat)
sat <- gsub("Somewhat Dissatisfied","3",sat)
sat <- gsub("Very Unsatisfied","1",sat)
sat <- gsub("Dissatisfied","2",sat)
sat <- gsub("Satisfied","6",sat)
sat1 <- as.numeric(sat)
sat1

##  [1] 7 6 6 6 5 6 6 6 7 7 7 7 7 6 7 6 6 6

mean(sat1)

## [1] 6.333333

# class 2 satisfaction
x$Q11

##  [1] "Very Satisfied" "Satisfied"      "Satisfied"      "Satisfied"     
##  [5] "Satisfied"      "Very Satisfied" "Satisfied"      "Satisfied"     
##  [9] "Very Satisfied" "Very Satisfied" "Very Satisfied" "Very Satisfied"
## [13] "Very Satisfied" "Satisfied"      "Very Satisfied" "Very Satisfied"
## [17] "Very Satisfied" "Very Satisfied"

sat <- x$Q11
sat <- sat[which(sat!="")]
sat <- sat[which(!is.na(sat))]
sat <- gsub("Very Satisfied","7",sat)
sat <- gsub("Somewhat Satisfied","5",sat)
sat <- gsub("Neutral","4",sat)
sat <- gsub("Somewhat Dissatisfied","3",sat)
sat <- gsub("Very Disatisfied","1",sat)
sat <- gsub("Dissatisfied","2",sat)
sat <- gsub("Satisfied","6",sat)
sat2 <- as.numeric(sat)
sat2

##  [1] 7 6 6 6 6 7 6 6 7 7 7 7 7 6 7 7 7 7

mean(sat2)

## [1] 6.611111

par(mfrow=c(2,1), cex=0.8)
hist(sat1,xlim=c(1,7))
hist(sat2,xlim=c(1,7))

wilcox.test(sat1,sat2)

## Warning in wilcox.test.default(sat1, sat2): cannot compute exact p-value with
## ties

## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  sat1 and sat2
## W = 122.5, p-value = 0.1597
## alternative hypothesis: true location shift is not equal to 0

t.test(sat1,sat2)

## 
##  Welch Two Sample t-test
## 
## data:  sat1 and sat2
## t = -1.5157, df = 33.071, p-value = 0.1391
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.65061069  0.09505513
## sample estimates:
## mean of x mean of y 
##  6.333333  6.611111

These results indicate average satisfaction was higher in Class 2 however it was not statistically significant.

Does the level of comfort with answering questions correlate with perceived utility or satisfaction?

how comfortable the students were answering the questions and either how useful, or how satisfied, they were with the classes (for either Class 1 or Class 2).

Class 1 comfort

# class 1: comfort = Q3, utility=Q4, satisfaction=Q5
x$Q3

##  [1] "Neither comfortable nor uncomfortable"
##  [2] "Moderately comfortable"               
##  [3] "Moderately comfortable"               
##  [4] "Extremely comfortable"                
##  [5] "Moderately comfortable"               
##  [6] ""                                     
##  [7] "Neither comfortable nor uncomfortable"
##  [8] "Moderately comfortable"               
##  [9] "Moderately comfortable"               
## [10] "Extremely comfortable"                
## [11] "Slightly uncomfortable"               
## [12] "Extremely comfortable"                
## [13] "Moderately comfortable"               
## [14] "Slightly comfortable"                 
## [15] "Moderately comfortable"               
## [16] "Moderately comfortable"               
## [17] "Moderately uncomfortable"             
## [18] "Moderately uncomfortable"

comf <- x$Q3
comf <- gsub("Extremely comfortable","7",comf)
comf <- gsub("Moderately comfortable","6",comf)
comf <- gsub("Slightly comfortable","5",comf)
comf <- gsub("Neither comfortable nor uncomfortable","4",comf)
comf <- gsub("Slightly uncomfortable","3",comf)
comf <- gsub("Moderately uncomfortable","2",comf)
comf1 <- gsub("Extremely uncomfortable","1",comf)
comf1

##  [1] "4" "6" "6" "7" "6" ""  "4" "6" "6" "7" "3" "7" "6" "5" "6" "6" "2" "2"

# class 1 utility
x$Q4

##  [1] "Extremely useful" "Very useful"      "Very useful"      "Very useful"     
##  [5] "Very useful"      "Very useful"      "Very useful"      "Extremely useful"
##  [9] "Very useful"      "Extremely useful" "Extremely useful" "Extremely useful"
## [13] "Extremely useful" "Very useful"      "Very useful"      "Extremely useful"
## [17] "Very useful"      "Very useful"

us <- x$Q4
us <- us[which(us!="")]
us <- us[which(!is.na(us))]
us <- gsub("Extremely useful","5",us)
us <- gsub("Very useful","4",us)
us <- gsub("Moderately useful","3",us)
us <- gsub("Slightly useful","2",us)
us1 <- gsub("Not at all useful","1",us)
us1

##  [1] "5" "4" "4" "4" "4" "4" "4" "5" "4" "5" "5" "5" "5" "4" "4" "5" "4" "4"

# class 1 satisfaction
x$Q5

##  [1] "Very Satisfied"     "Satisfied"          "Satisfied"         
##  [4] "Satisfied"          "Somewhat Satisfied" "Satisfied"         
##  [7] "Satisfied"          "Satisfied"          "Very Satisfied"    
## [10] "Very Satisfied"     "Very Satisfied"     "Very Satisfied"    
## [13] "Very Satisfied"     "Satisfied"          "Very Satisfied"    
## [16] "Satisfied"          "Satisfied"          "Satisfied"

sat <- x$Q5
sat <- gsub("Very Satisfied","7",sat)
sat <- gsub("Somewhat Satisfied","5",sat)
sat <- gsub("Neutral","4",sat)
sat <- gsub("Somewhat Dissatisfied","3",sat)
sat <- gsub("Very Dissatisfied","1",sat)
sat <- gsub("Dissatisfied","2",sat)
sat1 <- gsub("Satisfied","6",sat)
sat1

##  [1] "7" "6" "6" "6" "5" "6" "6" "6" "7" "7" "7" "7" "7" "6" "7" "6" "6" "6"

# check with another method because worries about NA values
z <- cbind(comf1,us1,sat1)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z

##       comf1 us1 sat1
##  [1,]     4   5    7
##  [2,]     6   4    6
##  [3,]     6   4    6
##  [4,]     7   4    6
##  [5,]     6   4    5
##  [6,]     4   4    6
##  [7,]     6   5    6
##  [8,]     6   4    7
##  [9,]     7   5    7
## [10,]     3   5    7
## [11,]     7   5    7
## [12,]     6   5    7
## [13,]     5   4    6
## [14,]     6   4    7
## [15,]     6   5    6
## [16,]     2   4    6
## [17,]     2   4    6

cor.test(z[,1],z[,2])

## 
##  Pearson's product-moment correlation
## 
## data:  z[, 1] and z[, 2]
## t = 0.69533, df = 15, p-value = 0.4975
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.3321476  0.6058919
## sample estimates:
##       cor 
## 0.1767077

cor.test(z[,1],z[,3])

## 
##  Pearson's product-moment correlation
## 
## data:  z[, 1] and z[, 3]
## t = 0.38844, df = 15, p-value = 0.7031
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.4000381  0.5538726
## sample estimates:
##        cor 
## 0.09979487

This result shows that there is no significant correlation between comfort and either usefulness or satisfaction in class 1. Now I can do the same for class 2.

# class 2: comfort = Q9, utility=Q10, satisfaction=Q11
# class 1: comfort = Q3, utility=Q4, satisfaction=Q5
x$Q9

##  [1] "Extremely comfortable"                
##  [2] "Extremely comfortable"                
##  [3] "Extremely comfortable"                
##  [4] "Moderately comfortable"               
##  [5] "Moderately comfortable"               
##  [6] "Slightly comfortable"                 
##  [7] "Moderately comfortable"               
##  [8] "Extremely comfortable"                
##  [9] "Extremely comfortable"                
## [10] "Extremely comfortable"                
## [11] "Extremely comfortable"                
## [12] "Neither comfortable nor uncomfortable"
## [13] "Extremely comfortable"                
## [14] "Extremely comfortable"                
## [15] "Extremely comfortable"                
## [16] "Extremely comfortable"                
## [17] "Moderately comfortable"               
## [18] "Moderately comfortable"

comf <- x$Q9
comf <- gsub("Extremely comfortable","1",comf)
comf <- gsub("Moderately comfortable","2",comf)
comf <- gsub("Slightly comfortable","3",comf)
comf <- gsub("Neither comfortable nor uncomfortable","4",comf)
comf <- gsub("Slightly uncomfortable","5",comf)
comf <- gsub("Moderately uncomfortable","6",comf)
comf2 <- gsub("Extremely uncomfortable","7",comf)
comf2

##  [1] "1" "1" "1" "2" "2" "3" "2" "1" "1" "1" "1" "4" "1" "1" "1" "1" "2" "2"

# class 1 utility
x$Q10

##  [1] "Extremely useful"  "Moderately useful" "Very useful"      
##  [4] "Very useful"       "Very useful"       "Extremely useful" 
##  [7] "Very useful"       "Very useful"       "Extremely useful" 
## [10] "Extremely useful"  "Extremely useful"  "Extremely useful" 
## [13] "Extremely useful"  "Very useful"       "Extremely useful" 
## [16] "Extremely useful"  "Extremely useful"  "Extremely useful"

us <- x$Q10
us <- us[which(us!="")]
us <- us[which(!is.na(us))]
us <- gsub("Extremely useful","1",us)
us <- gsub("Very useful","2",us)
us <- gsub("Moderately useful","3",us)
us <- gsub("Slightly useful","4",us)
us2 <- gsub("Not at all useful","5",us)
us2

##  [1] "1" "3" "2" "2" "2" "1" "2" "2" "1" "1" "1" "1" "1" "2" "1" "1" "1" "1"

# class 1 satisfaction
x$Q11

##  [1] "Very Satisfied" "Satisfied"      "Satisfied"      "Satisfied"     
##  [5] "Satisfied"      "Very Satisfied" "Satisfied"      "Satisfied"     
##  [9] "Very Satisfied" "Very Satisfied" "Very Satisfied" "Very Satisfied"
## [13] "Very Satisfied" "Satisfied"      "Very Satisfied" "Very Satisfied"
## [17] "Very Satisfied" "Very Satisfied"

sat <- x$Q11
sat <- gsub("Very Satisfied","1",sat)
sat <- gsub("Somewhat Satisfied","3",sat)
sat <- gsub("Neutral","4",sat)
sat <- gsub("Somewhat Dissatisfied","5",sat)
sat <- gsub("Very Unsatisfied","7",sat)
sat <- gsub("Dissatisfied","6",sat)
sat1 <- gsub("Satisfied","2",sat)
sat2

##  [1] 7 6 6 6 6 7 6 6 7 7 7 7 7 6 7 7 7 7

# check with another method because worries about NA values
z <- cbind(comf2,us2,sat2)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z

##       comf2 us2 sat2
##  [1,]     1   1    7
##  [2,]     1   3    6
##  [3,]     1   2    6
##  [4,]     2   2    6
##  [5,]     2   2    6
##  [6,]     3   1    7
##  [7,]     2   2    6
##  [8,]     1   2    6
##  [9,]     1   1    7
## [10,]     1   1    7
## [11,]     1   1    7
## [12,]     4   1    7
## [13,]     1   1    7
## [14,]     1   2    6
## [15,]     1   1    7
## [16,]     1   1    7
## [17,]     2   1    7
## [18,]     2   1    7

cor.test(z[,1],z[,2])

## 
##  Pearson's product-moment correlation
## 
## data:  z[, 1] and z[, 2]
## t = -0.65374, df = 16, p-value = 0.5226
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.5841744  0.3304602
## sample estimates:
##        cor 
## -0.1612948

cor.test(z[,1],z[,3])

## 
##  Pearson's product-moment correlation
## 
## data:  z[, 1] and z[, 3]
## t = 0.49097, df = 16, p-value = 0.6301
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.3658500  0.5570169
## sample estimates:
##      cor 
## 0.121829

This result shows that there is no significant correlation between comfort and either usefulness or satisfaction in class 2 either.

Is there a correlation between How comfortable the students were answering questions with their preferred style of class (Q13) or future class (Q14)

Not completely sure how to calculate this one. One solution would be to make a ratio of comfort for the two class types and then correlate the ratio with preferred class type. Responses to Q13 and Q14 were nearly identical so I will only analyse Q13 preferred class.

# class 1 comfort
x$Q3

##  [1] "Neither comfortable nor uncomfortable"
##  [2] "Moderately comfortable"               
##  [3] "Moderately comfortable"               
##  [4] "Extremely comfortable"                
##  [5] "Moderately comfortable"               
##  [6] ""                                     
##  [7] "Neither comfortable nor uncomfortable"
##  [8] "Moderately comfortable"               
##  [9] "Moderately comfortable"               
## [10] "Extremely comfortable"                
## [11] "Slightly uncomfortable"               
## [12] "Extremely comfortable"                
## [13] "Moderately comfortable"               
## [14] "Slightly comfortable"                 
## [15] "Moderately comfortable"               
## [16] "Moderately comfortable"               
## [17] "Moderately uncomfortable"             
## [18] "Moderately uncomfortable"

comf <- x$Q3
comf <- gsub("Extremely comfortable","7",comf)
comf <- gsub("Moderately comfortable","6",comf)
comf <- gsub("Slightly comfortable","5",comf)
comf <- gsub("Neither comfortable nor uncomfortable","4",comf)
comf <- gsub("Slightly uncomfortable","3",comf)
comf <- gsub("Moderately uncomfortable","2",comf)
comf1 <- gsub("Extremely uncomfortable","1",comf)

# class 2 comfort
x$Q9

##  [1] "Extremely comfortable"                
##  [2] "Extremely comfortable"                
##  [3] "Extremely comfortable"                
##  [4] "Moderately comfortable"               
##  [5] "Moderately comfortable"               
##  [6] "Slightly comfortable"                 
##  [7] "Moderately comfortable"               
##  [8] "Extremely comfortable"                
##  [9] "Extremely comfortable"                
## [10] "Extremely comfortable"                
## [11] "Extremely comfortable"                
## [12] "Neither comfortable nor uncomfortable"
## [13] "Extremely comfortable"                
## [14] "Extremely comfortable"                
## [15] "Extremely comfortable"                
## [16] "Extremely comfortable"                
## [17] "Moderately comfortable"               
## [18] "Moderately comfortable"

comf <- x$Q9
comf <- gsub("Extremely comfortable","7",comf)
comf <- gsub("Moderately comfortable","6",comf)
comf <- gsub("Slightly comfortable","5",comf)
comf <- gsub("Neither comfortable nor uncomfortable","4",comf)
comf <- gsub("Slightly uncomfortable","3",comf)
comf <- gsub("Moderately uncomfortable","2",comf)
comf2 <- gsub("Extremely uncomfortable","1",comf)

x$Q13

##  [1] "Class 2 (using Socrative)"    "Class 1 (traditional method)"
##  [3] "Class 2 (using Socrative)"    "Class 1 (traditional method)"
##  [5] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
##  [7] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
##  [9] ""                             ""                            
## [11] "Class 2 (using Socrative)"    ""                            
## [13] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
## [15] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
## [17] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"

pref <- x$Q13
pref <- gsub("Class 1 \\(traditional method\\)","1",pref)
pref <- gsub("Class 2 \\(using Socrative\\)","2",pref)
pref

##  [1] "2" "1" "2" "1" "2" "2" "2" "2" ""  ""  "2" ""  "2" "2" "2" "2" "2" "2"

z <- cbind(comf1,comf2,pref)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z <- as.data.frame(z)
# the ratio here is important because it is a negative scale
z$rat <- z$comf2 - z$comf1
z

##    comf1 comf2 pref rat
## 1      4     7    2   3
## 2      6     7    1   1
## 3      6     7    2   1
## 4      7     6    1  -1
## 5      6     6    2   0
## 6      4     6    2   2
## 7      6     7    2   1
## 8      3     7    2   4
## 9      6     7    2   1
## 10     5     7    2   2
## 11     6     7    2   1
## 12     6     7    2   1
## 13     2     6    2   4
## 14     2     6    2   4

cor.test(z$rat,z$pref)

## 
##  Pearson's product-moment correlation
## 
## data:  z$rat and z$pref
## t = 1.8516, df = 12, p-value = 0.08883
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.07891235  0.80151183
## sample estimates:
##       cor 
## 0.4714045

This result suggests there is a weak association between class comfort level and preference but it is not significant.

Is there a correlation between how useful the classes were and their preferred style of class (Q13) or future class (Q14)

# class 1 usefulness
x$Q4

##  [1] "Extremely useful" "Very useful"      "Very useful"      "Very useful"     
##  [5] "Very useful"      "Very useful"      "Very useful"      "Extremely useful"
##  [9] "Very useful"      "Extremely useful" "Extremely useful" "Extremely useful"
## [13] "Extremely useful" "Very useful"      "Very useful"      "Extremely useful"
## [17] "Very useful"      "Very useful"

us <- x$Q4
us <- gsub("Extremely useful","5",us)
us <- gsub("Very useful","4",us)
us <- gsub("Moderately useful","3",us)
us <- gsub("Slightly useful","2",us)
us1 <- gsub("Not at all useful","1",us)
us1

##  [1] "5" "4" "4" "4" "4" "4" "4" "5" "4" "5" "5" "5" "5" "4" "4" "5" "4" "4"

# class 2 usefulness
x$Q10

##  [1] "Extremely useful"  "Moderately useful" "Very useful"      
##  [4] "Very useful"       "Very useful"       "Extremely useful" 
##  [7] "Very useful"       "Very useful"       "Extremely useful" 
## [10] "Extremely useful"  "Extremely useful"  "Extremely useful" 
## [13] "Extremely useful"  "Very useful"       "Extremely useful" 
## [16] "Extremely useful"  "Extremely useful"  "Extremely useful"

us <- x$Q10
us <- gsub("Extremely useful","5",us)
us <- gsub("Very useful","4",us)
us <- gsub("Moderately useful","3",us)
us <- gsub("Slightly useful","2",us)
us2 <- gsub("Not at all useful","1",us)
us2

##  [1] "5" "3" "4" "4" "4" "5" "4" "4" "5" "5" "5" "5" "5" "4" "5" "5" "5" "5"

# preference
x$Q13

##  [1] "Class 2 (using Socrative)"    "Class 1 (traditional method)"
##  [3] "Class 2 (using Socrative)"    "Class 1 (traditional method)"
##  [5] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
##  [7] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
##  [9] ""                             ""                            
## [11] "Class 2 (using Socrative)"    ""                            
## [13] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
## [15] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
## [17] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"

pref <- x$Q13
pref <- gsub("Class 1 \\(traditional method\\)","1",pref)
pref <- gsub("Class 2 \\(using Socrative\\)","2",pref)
pref

##  [1] "2" "1" "2" "1" "2" "2" "2" "2" ""  ""  "2" ""  "2" "2" "2" "2" "2" "2"

z <- cbind(us1,us2,pref)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z <- as.data.frame(z)
# note the direction of the ratio
z$rat <- z$us2 - z$us1
z

##    us1 us2 pref rat
## 1    5   5    2   0
## 2    4   3    1  -1
## 3    4   4    2   0
## 4    4   4    1   0
## 5    4   4    2   0
## 6    4   5    2   1
## 7    4   4    2   0
## 8    5   4    2  -1
## 9    5   5    2   0
## 10   5   5    2   0
## 11   4   4    2   0
## 12   4   5    2   1
## 13   5   5    2   0
## 14   4   5    2   1
## 15   4   5    2   1

cor.test(z$rat,z$pref)

## 
##  Pearson's product-moment correlation
## 
## data:  z$rat and z$pref
## t = 1.5821, df = 13, p-value = 0.1376
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.1390833  0.7580416
## sample estimates:
##       cor 
## 0.4018072

This result suggests there is a weak negative association between perceived usefullness and preference but it is not significant.

Is there a correlation between student satisfaction with the classes with their preferred style of class (Q13) or future class (Q14)

# class 1 satisfaction
x$Q5

##  [1] "Very Satisfied"     "Satisfied"          "Satisfied"         
##  [4] "Satisfied"          "Somewhat Satisfied" "Satisfied"         
##  [7] "Satisfied"          "Satisfied"          "Very Satisfied"    
## [10] "Very Satisfied"     "Very Satisfied"     "Very Satisfied"    
## [13] "Very Satisfied"     "Satisfied"          "Very Satisfied"    
## [16] "Satisfied"          "Satisfied"          "Satisfied"

sat <- x$Q5
sat <- gsub("Very Satisfied","7",sat)
sat <- gsub("Somewhat Satisfied","5",sat)
sat <- gsub("Neutral","4",sat)
sat <- gsub("Somewhat Dissatisfied","3",sat)
sat <- gsub("Very Dissatisfied","1",sat)
sat <- gsub("Dissatisfied","2",sat)
sat1 <- gsub("Satisfied","6",sat)
sat1

##  [1] "7" "6" "6" "6" "5" "6" "6" "6" "7" "7" "7" "7" "7" "6" "7" "6" "6" "6"

# class 2 satisfaction
x$Q11

##  [1] "Very Satisfied" "Satisfied"      "Satisfied"      "Satisfied"     
##  [5] "Satisfied"      "Very Satisfied" "Satisfied"      "Satisfied"     
##  [9] "Very Satisfied" "Very Satisfied" "Very Satisfied" "Very Satisfied"
## [13] "Very Satisfied" "Satisfied"      "Very Satisfied" "Very Satisfied"
## [17] "Very Satisfied" "Very Satisfied"

sat <- x$Q11
sat <- gsub("Very Satisfied","7",sat)
sat <- gsub("Somewhat Satisfied","5",sat)
sat <- gsub("Neutral","4",sat)
sat <- gsub("Somewhat Dissatisfied","3",sat)
sat <- gsub("Very Dissatisfied","1",sat)
sat <- gsub("Dissatisfied","2",sat)
sat2 <- gsub("Satisfied","6",sat)
sat2

##  [1] "7" "6" "6" "6" "6" "7" "6" "6" "7" "7" "7" "7" "7" "6" "7" "7" "7" "7"

# preference
x$Q13

##  [1] "Class 2 (using Socrative)"    "Class 1 (traditional method)"
##  [3] "Class 2 (using Socrative)"    "Class 1 (traditional method)"
##  [5] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
##  [7] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
##  [9] ""                             ""                            
## [11] "Class 2 (using Socrative)"    ""                            
## [13] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
## [15] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
## [17] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"

pref <- x$Q13
pref <- gsub("Class 1 \\(traditional method\\)","1",pref)
pref <- gsub("Class 2 \\(using Socrative\\)","2",pref)
pref

##  [1] "2" "1" "2" "1" "2" "2" "2" "2" ""  ""  "2" ""  "2" "2" "2" "2" "2" "2"

z <- cbind(sat1,sat2,pref)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z <- as.data.frame(z)
# note the direction of the ratio
z$rat <- z$sat2 - z$sat1
z

##    sat1 sat2 pref rat
## 1     7    7    2   0
## 2     6    6    1   0
## 3     6    6    2   0
## 4     6    6    1   0
## 5     5    6    2   1
## 6     6    7    2   1
## 7     6    6    2   0
## 8     6    6    2   0
## 9     7    7    2   0
## 10    7    7    2   0
## 11    6    6    2   0
## 12    7    7    2   0
## 13    6    7    2   1
## 14    6    7    2   1
## 15    6    7    2   1

cor.test(z$rat,z$pref)

## 
##  Pearson's product-moment correlation
## 
## data:  z$rat and z$pref
## t = 1.0408, df = 13, p-value = 0.3169
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.2738154  0.6913838
## sample estimates:
##       cor 
## 0.2773501

This result suggests there is a weak association between satisfaction and preference but it is not significant.

Is there an association between differential participation level and preferred class type?

# participation for class 1
part <- x$Q2
part <- gsub('Yes \\(more than one question\\)',"2",part)
part1 <- gsub("No","0",part)
part1

##  [1] "0" "0" "2" "0" "0" "0" "0" ""  "0" "0" "0" "0" "2" "0" "2" "0" ""  "0"

# participation for class 2
part <- x$Q8
part <- gsub('Yes \\(but only a few of the questions\\)',"1",part)
part <- gsub('Yes \\(all or almost all of the questions\\)',"2",part)
part2 <- gsub("No","0",part)
part2

##  [1] "2" "2" "2" "1" "0" "1" "1" ""  "2" "2" "2" "0" "2" "1" "2" "1" ""  "2"

# preference
x$Q13

##  [1] "Class 2 (using Socrative)"    "Class 1 (traditional method)"
##  [3] "Class 2 (using Socrative)"    "Class 1 (traditional method)"
##  [5] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
##  [7] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
##  [9] ""                             ""                            
## [11] "Class 2 (using Socrative)"    ""                            
## [13] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
## [15] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"   
## [17] "Class 2 (using Socrative)"    "Class 2 (using Socrative)"

pref <- x$Q13
pref <- gsub("Class 1 \\(traditional method\\)","1",pref)
pref <- gsub("Class 2 \\(using Socrative\\)","2",pref)
pref <- as.numeric(pref)
pref

##  [1]  2  1  2  1  2  2  2  2 NA NA  2 NA  2  2  2  2  2  2

z <- cbind(part1,part2,pref)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z <- as.data.frame(z)

# note the direction of the ratio
z$rat <- z$part2 - z$part1
z

##    part1 part2 pref rat
## 1      0     2    2   2
## 2      0     2    1   2
## 3      2     2    2   0
## 4      0     1    1   1
## 5      0     0    2   0
## 6      0     1    2   1
## 7      0     1    2   1
## 8      0     2    2   2
## 9      2     2    2   0
## 10     0     1    2   1
## 11     2     2    2   0
## 12     0     1    2   1
## 13     0     2    2   2

cor.test(z$rat,z$pref)

## 
##  Pearson's product-moment correlation
## 
## data:  z$rat and z$pref
## t = -0.93664, df = 11, p-value = 0.3691
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.7156049  0.3283803
## sample estimates:
##        cor 
## -0.2717787

There was no clear association between participation level and the preferred class.

Is there an association between level of comfort and participation in Revision Class 1?

Class 1 - level of comfort (Q3)

Class 1 - participation (Q2)

# participation for class 1
part <- x$Q2
part <- gsub('Yes \\(more than one question\\)',"2",part)
part <- gsub("No","0",part)
part1 <- as.numeric(part)
part1

##  [1]  0  0  2  0  0  0  0 NA  0  0  0  0  2  0  2  0 NA  0

# class 1 comfort
x$Q3

##  [1] "Neither comfortable nor uncomfortable"
##  [2] "Moderately comfortable"               
##  [3] "Moderately comfortable"               
##  [4] "Extremely comfortable"                
##  [5] "Moderately comfortable"               
##  [6] ""                                     
##  [7] "Neither comfortable nor uncomfortable"
##  [8] "Moderately comfortable"               
##  [9] "Moderately comfortable"               
## [10] "Extremely comfortable"                
## [11] "Slightly uncomfortable"               
## [12] "Extremely comfortable"                
## [13] "Moderately comfortable"               
## [14] "Slightly comfortable"                 
## [15] "Moderately comfortable"               
## [16] "Moderately comfortable"               
## [17] "Moderately uncomfortable"             
## [18] "Moderately uncomfortable"

comf <- x$Q3
comf <- gsub("Extremely comfortable","7",comf)
comf <- gsub("Moderately comfortable","6",comf)
comf <- gsub("Slightly comfortable","5",comf)
comf <- gsub("Neither comfortable nor uncomfortable","4",comf)
comf <- gsub("Slightly uncomfortable","3",comf)
comf <- gsub("Moderately uncomfortable","2",comf)
comf <- gsub("Extremely uncomfortable","1",comf)
comf1 <- as.numeric(comf)
comf1

##  [1]  4  6  6  7  6 NA  4  6  6  7  3  7  6  5  6  6  2  2

z <- cbind(comf1,part1)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z <- as.data.frame(z)
z

##    comf1 part1
## 1      4     0
## 2      6     0
## 3      6     2
## 4      7     0
## 5      6     0
## 6      4     0
## 7      6     0
## 8      7     0
## 9      3     0
## 10     7     0
## 11     6     2
## 12     5     0
## 13     6     2
## 14     6     0
## 15     2     0

cor.test(z$comf1,z$part1)

## 
##  Pearson's product-moment correlation
## 
## data:  z$comf1 and z$part1
## t = 0.76169, df = 13, p-value = 0.4598
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  -0.3417563  0.6501197
## sample estimates:
##       cor 
## 0.2066918

There is no correlation, likely because there was too little participation.

Is there an association between level of comfort and participation in Revision Class 2?

Class 2 - level of comfort (Q9)

Class 2 - participation (Q8)

# participation for class 2
part <- x$Q8
part <- gsub('Yes \\(but only a few of the questions\\)',"1",part)
part <- gsub('Yes \\(all or almost all of the questions\\)',"2",part)
part <- gsub("No","0",part)
part2 <- as.numeric(part)
part2

##  [1]  2  2  2  1  0  1  1 NA  2  2  2  0  2  1  2  1 NA  2

# class 2 comfort
x$Q9

##  [1] "Extremely comfortable"                
##  [2] "Extremely comfortable"                
##  [3] "Extremely comfortable"                
##  [4] "Moderately comfortable"               
##  [5] "Moderately comfortable"               
##  [6] "Slightly comfortable"                 
##  [7] "Moderately comfortable"               
##  [8] "Extremely comfortable"                
##  [9] "Extremely comfortable"                
## [10] "Extremely comfortable"                
## [11] "Extremely comfortable"                
## [12] "Neither comfortable nor uncomfortable"
## [13] "Extremely comfortable"                
## [14] "Extremely comfortable"                
## [15] "Extremely comfortable"                
## [16] "Extremely comfortable"                
## [17] "Moderately comfortable"               
## [18] "Moderately comfortable"

comf <- x$Q9
comf <- gsub("Extremely comfortable","7",comf)
comf <- gsub("Moderately comfortable","6",comf)
comf <- gsub("Slightly comfortable","5",comf)
comf <- gsub("Neither comfortable nor uncomfortable","4",comf)
comf <- gsub("Slightly uncomfortable","3",comf)
comf <- gsub("Moderately uncomfortable","2",comf)
comf <- gsub("Extremely uncomfortable","1",comf)
comf2 <- as.numeric(comf)
comf2

##  [1] 7 7 7 6 6 5 6 7 7 7 7 4 7 7 7 7 6 6

z <- cbind(comf2,part2)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z <- as.data.frame(z)
z

##    comf2 part2
## 1      7     2
## 2      7     2
## 3      7     2
## 4      6     1
## 5      6     0
## 6      5     1
## 7      6     1
## 8      7     2
## 9      7     2
## 10     7     2
## 11     4     0
## 12     7     2
## 13     7     1
## 14     7     2
## 15     7     1
## 16     6     2

cor.test(z$comf2,z$part2)

## 
##  Pearson's product-moment correlation
## 
## data:  z$comf2 and z$part2
## t = 3.8015, df = 14, p-value = 0.001945
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.3355290 0.8929379
## sample estimates:
##       cor 
## 0.7126962

There is a strong correlation between comfort level and participation (p=0.002).

Is there an association between usefulness and student satisfaction in Revision Class 1?

Class 1 - usefulness (Q4)

Class 1 - satisfaction (Q5)

# class 1 utility
x$Q4

##  [1] "Extremely useful" "Very useful"      "Very useful"      "Very useful"     
##  [5] "Very useful"      "Very useful"      "Very useful"      "Extremely useful"
##  [9] "Very useful"      "Extremely useful" "Extremely useful" "Extremely useful"
## [13] "Extremely useful" "Very useful"      "Very useful"      "Extremely useful"
## [17] "Very useful"      "Very useful"

us <- x$Q4
us <- gsub("Extremely useful","5",us)
us <- gsub("Very useful","4",us)
us <- gsub("Moderately useful","3",us)
us <- gsub("Slightly useful","2",us)
us <- gsub("Not at all useful","1",us)
us1 <- as.numeric(us)
us1

##  [1] 5 4 4 4 4 4 4 5 4 5 5 5 5 4 4 5 4 4

# class 1 satisfaction
x$Q5

##  [1] "Very Satisfied"     "Satisfied"          "Satisfied"         
##  [4] "Satisfied"          "Somewhat Satisfied" "Satisfied"         
##  [7] "Satisfied"          "Satisfied"          "Very Satisfied"    
## [10] "Very Satisfied"     "Very Satisfied"     "Very Satisfied"    
## [13] "Very Satisfied"     "Satisfied"          "Very Satisfied"    
## [16] "Satisfied"          "Satisfied"          "Satisfied"

sat <- x$Q5
sat <- gsub("Very Satisfied","7",sat)
sat <- gsub("Somewhat Satisfied","5",sat)
sat <- gsub("Neutral","4",sat)
sat <- gsub("Somewhat Dissatisfied","3",sat)
sat <- gsub("Very Unsatisfied","1",sat)
sat <- gsub("Dissatisfied","2",sat)
sat <- gsub("Satisfied","6",sat)
sat1 <- as.numeric(sat)
sat1

##  [1] 7 6 6 6 5 6 6 6 7 7 7 7 7 6 7 6 6 6

z <- cbind(us1,sat1)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z <- as.data.frame(z)
z

##    us1 sat1
## 1    5    7
## 2    4    6
## 3    4    6
## 4    4    6
## 5    4    5
## 6    4    6
## 7    4    6
## 8    5    6
## 9    4    7
## 10   5    7
## 11   5    7
## 12   5    7
## 13   5    7
## 14   4    6
## 15   4    7
## 16   5    6
## 17   4    6
## 18   4    6

cor.test(z$us1,z$sat1)

## 
##  Pearson's product-moment correlation
## 
## data:  z$us1 and z$sat1
## t = 2.4762, df = 16, p-value = 0.02483
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.07887395 0.79730084
## sample estimates:
##       cor 
## 0.5263614

There was a positive association between perceived utility and satisfaction (p=0.025).

Is there an association between usefulness and student satisfaction in Revision Class 1?

Class 2 - usefulness (Q10)

Class 2 - satisfaction (Q11)

# class 2 utility
x$Q10

##  [1] "Extremely useful"  "Moderately useful" "Very useful"      
##  [4] "Very useful"       "Very useful"       "Extremely useful" 
##  [7] "Very useful"       "Very useful"       "Extremely useful" 
## [10] "Extremely useful"  "Extremely useful"  "Extremely useful" 
## [13] "Extremely useful"  "Very useful"       "Extremely useful" 
## [16] "Extremely useful"  "Extremely useful"  "Extremely useful"

us <- x$Q10
us <- gsub("Extremely useful","5",us)
us <- gsub("Very useful","4",us)
us <- gsub("Moderately useful","3",us)
us <- gsub("Slightly useful","2",us)
us <- gsub("Not at all useful","1",us)
us2 <- as.numeric(us)
us2

##  [1] 5 3 4 4 4 5 4 4 5 5 5 5 5 4 5 5 5 5

# class 2 satisfaction
x$Q11

##  [1] "Very Satisfied" "Satisfied"      "Satisfied"      "Satisfied"     
##  [5] "Satisfied"      "Very Satisfied" "Satisfied"      "Satisfied"     
##  [9] "Very Satisfied" "Very Satisfied" "Very Satisfied" "Very Satisfied"
## [13] "Very Satisfied" "Satisfied"      "Very Satisfied" "Very Satisfied"
## [17] "Very Satisfied" "Very Satisfied"

sat <- x$Q11
sat <- gsub("Very Satisfied","7",sat)
sat <- gsub("Somewhat Satisfied","5",sat)
sat <- gsub("Neutral","4",sat)
sat <- gsub("Somewhat Dissatisfied","3",sat)
sat <- gsub("Very Unsatisfied","1",sat)
sat <- gsub("Dissatisfied","2",sat)
sat <- gsub("Satisfied","6",sat)
sat2 <- as.numeric(sat)
sat2

##  [1] 7 6 6 6 6 7 6 6 7 7 7 7 7 6 7 7 7 7

z <- cbind(us2,sat2)
z <- apply(z,2,as.numeric)
z <- z[!is.na(rowSums(z)),]
z <- as.data.frame(z)
z

##    us2 sat2
## 1    5    7
## 2    3    6
## 3    4    6
## 4    4    6
## 5    4    6
## 6    5    7
## 7    4    6
## 8    4    6
## 9    5    7
## 10   5    7
## 11   5    7
## 12   5    7
## 13   5    7
## 14   4    6
## 15   5    7
## 16   5    7
## 17   5    7
## 18   5    7

cor.test(z$us2,z$sat2)

## 
##  Pearson's product-moment correlation
## 
## data:  z$us2 and z$sat2
## t = 10.213, df = 16, p-value = 2.048e-08
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.8212767 0.9744068
## sample estimates:
##       cor 
## 0.9311257

There was a strong association between perceived utility and satisfaction (p=2.0e-8).

Conclusions

There was a massive difference in student participation between class types, with the Socrative app resulting in a much higher participation rate. Students were more comfortable using the Socrative app as compared to traditional/conventional methods. Traditional and socrative classes were viewed as useful with only a small difference between them. When students passively attend class they may have a false sense of security because they lack chances to benchmark own performance with formative assessment. Similarly there was no significant difference in satisfaction between class types. There was no significant correlation between comfort with answering questions and perceived utility or satisfaction in either class. Students that participated more in class 2 as compared to class 1 tended to prefer Socrative over traditional methods (p=0.09). There was no correlation between differential utility and preference. There was no correlation between differential satisfaction and preference. While the simple comparisons gave a clear result, the correlation analyses likely require a larger sample size. Correlation analysis between level of comfort and participation shows no assocation in class 1 but strong association in class 2. Perceived utility was correlated with student satisfaction in both class 1 and 2.

Session information

sessionInfo()

## R version 4.0.2 (2020-06-22)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Ubuntu 18.04.4 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
## LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so
## 
## locale:
##  [1] LC_CTYPE=en_AU.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_AU.UTF-8        LC_COLLATE=en_AU.UTF-8    
##  [5] LC_MONETARY=en_AU.UTF-8    LC_MESSAGES=en_AU.UTF-8   
##  [7] LC_PAPER=en_AU.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C       
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## loaded via a namespace (and not attached):
##  [1] compiler_4.0.2  magrittr_1.5    tools_4.0.2     htmltools_0.5.0
##  [5] yaml_2.2.1      stringi_1.4.6   rmarkdown_2.3   knitr_1.29     
##  [9] stringr_1.4.0   xfun_0.15       digest_0.6.25   rlang_0.4.6    
## [13] evaluate_0.14

Stats analysis for Mat

Mark Ziemann

2020-07-24

Intro

Questions

Q1. “For Revision Class 1 (on Tuesday 6th August), did you:”

Q2. “If you attended Revision Class 1, did you answer any questions aloud in front of the class?”

Q3. “How comfortable did you feel to answer questions in Revision Class 1?”

Q4. “How would you rate the usefulness of Revision Class 1 for understanding the topics covered?”

Q5. “How would you rate your overall satisfaction with Revision Class 1?”

Q6. “Any comments about Revision Class 1 and answering questions aloud during the class?”

Q7. “For Revision Class 2 (Monday 26th August, using the Socrative App), did you:”

Q8. “If you attended Revision Class 2, did you use the Socrative App to answer any questions?”

Q9. “How comfortable did you feel to answer questions in Revision Class 2 using the Socrative App?”

Q10. “How would you rate the usefulness of Revision Class 2 for understanding the topics covered?”

Q11. “How would you rate your overall satisfaction with Revision Class 2?”

Q12. “Any comments about Revision Class 2 and using the Socrative App?”

Q13. “Overall, did you prefer Revision Class 1 (traditional method of answering questions aloud during the class), or Revision class 2 (using the Socrative App to answer questions)?”

Q14. “For future revision classes, which method would you prefer to be used?”

Load the data

Matt’s direction for statistical analysis

Was there a difference in participation between traditional and socrative classes?

Were students more comfortable to participate in answering questions in traditional or Socrative classes?

Did students perceive traditional or Socrative classes to be more useful?

Were students more satisfied with traditional or Socrative classes?

Does the level of comfort with answering questions correlate with perceived utility or satisfaction?

Is there a correlation between How comfortable the students were answering questions with their preferred style of class (Q13) or future class (Q14)

Is there a correlation between how useful the classes were and their preferred style of class (Q13) or future class (Q14)

Is there a correlation between student satisfaction with the classes with their preferred style of class (Q13) or future class (Q14)

Is there an association between differential participation level and preferred class type?

Is there an association between level of comfort and participation in Revision Class 1?

Is there an association between level of comfort and participation in Revision Class 2?

Is there an association between usefulness and student satisfaction in Revision Class 1?

Is there an association between usefulness and student satisfaction in Revision Class 1?

Conclusions

Session information