Interpretation Protocol (live Qualtrics surveys for WE1S developers)

Interpretation Protocol files also deposited in Zenodo: doi: 10.5281/zenodo.4940170

The WE1S Interpretation Protocol for topic models (currently in version 2.0) is implemented most functionally as a modular set of Qualtrics surveys with declared steps for observing and drawing conclusions from topic models, and then recording results through the survey. The survey modules are designed to be mixed-and-matched in sequence or in parallel depending on the needs of a particular research problem.

Below are links to the live surveys. These are accessible in their live form only by WE1S developers using the Qualtrics survey platform through a UC Santa Barbara institutional license. Users at other institutions with their own access to the Qualtrics platform may import our surveys in the form of QSF export/import files. (Our Interpretation Protocol is tailored with specific references to our project, but suggests how other users can extend and revise the paradigm for their own projects.)

Once an instance of a survey is started, it can only be used by one person (i.e., computer) at a time in a unique session. Teams working on a survey together might collaborate by using our Word .docx versions of Interpretation Protocol modules to draft responses together.

Each Qualtrics survey finishes with an option to request an emailed copy of answers. (Note: Once a survey is started, it must be completed — even if only with incomplete answers — before being restarted. Users can also conduct trial runs of surveys by checking the box for “trial” at the beginning of each module., which allows trials to be set aside during Qualtrics analyses of results.)

The functional advantages of using the Interpretation Protocol in its Qualtrics version include the conditional, branching logic that some of the modules utilize and, most importantly, Qualtrics’s processes for analyzing and reporting on results.

Live versions of modules of the Interpretation Protocol

Getting an overview of a topic model:

Specific research tasks:

Final research steps:

Module 0
User Training
Module 0...
Module 1
Overview of model
Module 1...
Module 2
Representative topics
Module 2...
Module 3.a
Analyze a topic
Module 3.a...
Module 3.b
Analyze a cluster of topics
Module 3.b...
Module 3.c
Analyze a keyword
Module 3.c...
Module 4.a
Compare sets of topics (multiple topics)
Module 4.a...
Module 4.c
Compare two keywords
Module 4.c...
Module 5
Compare two parts of corpus
     * part to whole
     * 2 metadata sets
     * 2 time ranges
     * compare to "random" corpus
Module 5...
Module Z
Add-on steps for collaborative interpretation
Module Z...
Module 7
Analysis & synthesis of interpretation results
Module 7...
Report Module --
Instructions for writing report
Report Module --...
= Created
= Created
Module 6
Compare two different
topic models
Module 6...
Module X
Make & document your
own workflow
Module X...
(use Google Doc
research report template
Module Y
Document use of additional methods and tools
Module Y...
Viewer does not support full SVG 1.1
Flowchart of existing WE1S Interpretation Protocol modules. (Modules 0 and 5 are yet to created.) Click on diagram for a larger image.