These are example datasets used as part of creating the Project Aristo
Aristo science exams
Science exam questions guide our research into Question Answering for 4th-grade biology exams. They are derived from actual 4th-grade biology exams.
: These question sets are derived from the New York State Education Department, Grade 4 Elementary-Level Science Test (http://www.nysedregents.org/Grade4/Science/home.html
; accessed July 2014) and science questions published by Help Teaching
These questions guide our research into Question Answering for arithmetic exams. Focus is on 4th-grade level. Example: "Sandy has 10 books , Benny has 24 books , and Tim has 33 books . How many books do they have together?"
: AI2 & University of Washington (Hannaneh Hajishirzi
) collaboration on arithmetic exam Question Answering.
Biology How/Why Corpus
This dataset consists of 185 "how" and 193 "why" biology questions authored by a domain expert, with one or more gold answer passages identified in an undergraduate textbook. The expert was not constrained in any way during the annotation process, so gold answers might be smaller than a paragraph or span multiple paragraphs. This dataset was used for the question-answering system described in "DiscourseComplements Lexical Semantics for Non-factoid Answer Reranking"
: AI2 & University of Arizona (Mihai Surdeanu)
An understanding of co-reference (i.e. multiple references to the same thing) is necessary in order to understand the meaning of a text. This dataset is an analysis of co-reference types occurring in 4th-grade biology textbooks.
: This analysis was performed by AI2 based on the New York State Education Department, Grade 4 Elementary-Level Science Test (http://www.nysedregents.org/Grade4/Science/home.html
; accessed July 2014).
Odd-Man-Out Problem Set
This collection consists of four sets of "odd-man-out" puzzles. There are two collections of "common noun" puzzles, where the answer options are largely common nouns, and two collections of "proper noun" puzzles, where the answers options are largely
proper nouns. Each collection contains approximately 100 puzzles.
: AI2 & University of Washington collaboration on geometry exam Question Answering.
These questions guide our research into Question Answering for geometry exams. Focus is on 4th-grade level. Example (note: diagrams included in data file): "In circle O, diameter AB is perpendicular to chord CD at E. If CD = 8 and BE = 2, find AE.
: This data set was produced at AI2; categories are taken from the card game Anomia, which was used to drive the puzzle generation process.
Vocabulary used in questions may differ from that of sources contributing to our Question Answering knowledge base. Relevant paraphrases like these help the QA system understand connections between question vocabulary and knowledge base vocabulary. This dataset is an example of analysis done by AI2 intern Ellie Pavlick
. example "get a better look at/view in more detail"
: Subset of AI2 intern Ellie Pavlick’s analysis of PPDB
paraphrases relevant to 4th-grade biology exams.
This dataset was used to train a system to automatically extract process models from paragraphs that describe processes. The dataset consists of 200 paragraphs that describe biological processes. Each paragraph is annotated with its process structure, and accompanied by a few multiple-choice questions about the process. Each question has two possible answers of which exactly one is correct.
The dataset contains three files:
- bioprocess-bank-questions.tar.gz: There is an xml file for each paragraph containing the paragraph ID, the questions and answers.
- process-bank-structures-train.tar.gz: These are the structure annotations used for training our structure predictor. Each paragraph has two files - one containing the text and one containing the annotation. This is standard BRAT format (http://brat.nlplab.org/).
- process-bank-structures-test.tar.gz: These are structure annotations used for testing. They are also in BRAT format.
: AI2 & Stanford University (Jonathan Berant