Skip to Main Content
 

Global Search Box

 
 
 
 

Files

File List


Thesis-Final.pdf (736.29 KB)

ETD Abstract Container

Abstract Header

QED: A Fact Verification and Evidence Support System

Luken, Jackson
http://rave.ohiolink.edu/etdc/view?acc_num=osu1555074124008897

Abstract Details

2019, Master of Science, Ohio State University, Computer Science and Engineering.
In today's online world with millions of voices all making numerous claims, it can be difficult to tell who is telling the truth and who is lying (or maybe just wrong.) Even if the correct information is available, it can seem insurmountable to try and check every supposedly true statement made. Additionally, the fact that it is so easy to quickly spread new information can make fact-checking existing assertions a sisyphean task. It is for this reason that an automated solution is needed to combat the flood of falsehoods found online. This work explores the viability of an automated system and its components which could be used to verify any claims, given a database of facts. The system is based on a heuristics-based model to identify key words and phrases within a claim, and then match it with relevant document titles from a corpus of over five million Wikipedia articles. After identifying noun phrases and named entities within each sentence of the documents, the claim is matched with evidence that could potentially either verify or refute it. Finally, each piece of evidence is entered into an inference classifier alongside the claim and scored as to how much it either verifies or refutes it. The scores are tallied and the "best" classification label (or potentially simply "not enough info" if there is no sufficiently scored evidence to support or refute the claim) is assigned to the claim alongside the supporting evidence. With this system, we were able to achieve extremely promising results when classifying test data. It was tested against the FEVER shared task dataset and evaluated using their scoring system. Our F1 score in retrieving relevant evidence was 58.54%, beating the baseline of 18.66%. When evaluated based on whether we could correctly classify the claim and retrieve at least one relevant piece of evidence, we achieved a score of 43.22%, beating the baseline of 27.71% and ranking 7th out of 24 teams in the shared task.
Marie-Catherine de Marneffe (Advisor)
Alan Ritter (Committee Member)
48 p.
Computer Science
FEVER; Natural Language Processing; Fact Verification; Computer Science; Machine Learning

Recommended Citations

Citations

  • Luken, J. (2019). QED: A Fact Verification and Evidence Support System [Master's thesis, Ohio State University]. OhioLINK Electronic Theses and Dissertations Center. http://rave.ohiolink.edu/etdc/view?acc_num=osu1555074124008897

    APA Style (7th edition)

  • Luken, Jackson. QED: A Fact Verification and Evidence Support System. 2019. Ohio State University, Master's thesis. OhioLINK Electronic Theses and Dissertations Center, http://rave.ohiolink.edu/etdc/view?acc_num=osu1555074124008897.

    MLA Style (8th edition)

  • Luken, Jackson. "QED: A Fact Verification and Evidence Support System." Master's thesis, Ohio State University, 2019. http://rave.ohiolink.edu/etdc/view?acc_num=osu1555074124008897

    Chicago Manual of Style (17th edition)

osu1555074124008897
373
© 2019, all rights reserved.
This open access ETD is published by The Ohio State University and OhioLINK.