|
|
Study |
Technique |
Data |
Sample size |
Analysis |
Potential Strength of Evidence |
Cost of Planning |
Cost of Execution |
Cost of Analysis |
|
Simon (i) |
Suite of “predictive” tests |
·
Find/create stuff ·
Create an instrument ·
Check against results in
course (aka marks/grades) |
numeric |
ca. 20 |
statistical |
low |
VERY HIGH |
low |
low |
|
Simon (ii) |
Syntax/synthesis continuum |
·
Collect assessments (from
many courses) ·
Categorize |
very qual. |
10-20 |
build consensus on model |
depends |
med-low |
VERY HIGH |
VERY HIGH |
|
Ken |
Musical ability as a predictor of
programming success |
·
Aural testing (software) ·
Survey ·
Music grades ·
CS grades |
numeric |
20 then 100-200 |
statistical |
good |
HUGE |
low |
low |
|
John |
Peer Assessment |
·
Marked coursework ·
Interviews ·
Surveys ·
Audit trails (software) |
numeric transcript automatic |
100x12 |
statistical qual
(survey & traces) |
contextualised – but can be high |
“not un-manageably high” |
med |
med |
|
Quintin |
Handset usage as predictor |
·
Lab/tutorial attendance ·
Handset usage in lectures ·
Grades |
numeric (naturally occurring) |
longitudinal 170/250/330 |
statistical |
good |
low |
low |
low |
|
Margaret |
Tablet PCs and groupwork |
·
Interviews @ 3 points (20) ·
Spyware ·
Postings to newsgroups |
numeric qual
– transcripts, newsgroups |
20/60 |
counts statistical transcripts & survey |
low-med |
high |
high |
high |
|
Jodi |
Classroom learning and Tablet PCs |
·
CoP
framework ·
Teacher logs ·
Attendance / surveys / drop
out rates ·
Interviews |
numeric qual –
theoretical stance |
70/30 |
qual. |
med |
was huge: now OK |
med |
high |
|
Denise |
Epistemological Beliefs (EB) in
capstone project course (collaborative) |
·
EB survey (existing instrument) ·
Selected interviews ·
Grades etc. |
numeric transcripts |
2 insts. survey: 80/30 interview: 9x2 |
statistical qual. |
good |
low (opportunistic) |
med |
med |
|
Ray (i) |
|
Staff are subjects (and maybe
some senior studes) ·
take test ·
think-aloud ·
doodles |
transcripts artefacts |
3 |
qual |
med |
none |
low – med |
low – med |
|
Ray (ii) |
“Authentic” programming tasks |
·
take test ·
think-aloud ·
doodles |
transcripts artefacts & performance data |
circa 100 (mi-mn) |
statistical qual. |
good |
med |
high |
high |
|
Michael |
Teaching explicit problem-solving |
Theoretical framework (goal-plan
analysis) ·
control group ·
pre-test, post-test ·
exit interviews |
numeric (& teacher logs) |
10-20 balanced groups |
·
goal-plan analysis on
student solutions ·
statistical |
sceptical |
high |
HIGHER |
med-low |
|
Ilona |
Variation of understanding
software developers have of Business Information Systems Analysis &
Design |
Phenomenographic |
·
interview transcripts ·
historical descriptions |
25 |
phenomenographic (interpretative,
qual.) |
phenomenographic
strength |
high |
high (Ilona
says med) |
very, very high |