This seminar introduces fundamental methods for both qualitative and quantitative research in information studies. Additionally, the seminar introduces the student to established and emerging areas of scholarly research in Schools of Information to encourage her to identify a personal research agenda. The seminar is organized in two main parts: the first part introduces relevant research methods (quantitative and qualitative), whereas the second part overviews specific research directions currently active in the School of Information. The second part of the seminar will be covered by guest faculty who research in each of the covered areas.
1. Quantitative research methods (4 to 5 weeks)
a. Exploratory data analysis. How to visualize and summarize data to uncover its hidden structure. Elementary data mining techniques.
b. Experiment design. Why do experiments? Experimental control of programs; control of human/computer systems; using panels of human judges to evaluate system performance; blinding the panel to the source of answers. Spurious effects and biased sampling. Pilot experiments. Guidelines and tips.
c. Hypothesis testing and confidence intervals. The standard error of a statistic and sample size. Sampling distribution of the mean. Testing hypotheses about means. Confidence intervals. The difference between significance and meaning. Effect size. Should we care about significance tests when samples are enormous? Non-parametric vs. parametric hypothesis testing methods.
d. Performance Assessment. Common performance measures. “Gold standard” corpora. Testing on training sets. Cross-validation. Multiple testing problems. Variability, predictive power. So you got a significant result, but is it meaningful? Metrics such as predictive power that augment statistically significant differences in performance.
e. Generalization, the aim of science. The behavior of a system on a single dataset isn’t as interesting as what we can say about its behavior on a large class of datasets, or what we can say about equivalence classes of systems. Computer science has several strategies for moving from specific to general. One is to analytically classify problems by their complexity, but empirically there is much variance among problems in each analytical class. Another strategy is to develop corpora that are thought on empirical grounds to be representative of various classes of applied problems. The empirical generalization strategy and how it ties in with theoretical analysis.
2. Qualitative research methods (4 weeks)
i. Interviews (to include the difference between qualitative interviewing and interviews for a quantitative study)
ii. Observations (empirical examination of people, texts, or processes)
d. Tools for qualitative research: Qualtrix, LibQual, Atlas-TI, nvivo.
3. Research dissemination (2 weeks)
a. Writing papers, writing styles (deductive or inductive logic)
b. Reviewing papers
c. Responding to peer reviews
d. Academic presentations
4. Research ethics (1 week or less)
a. Protecting human research participants; Institutional Review Board (IRB)
5. Research topics in the School of Information Survey of active School Research (4 weeks)
a. Quantitative research in computational intelligence
b. Quantitative and qualitative research in digital arts
c. Quantitative and qualitative research in digital humanities
d. Philosophical analysis