Novel contexts, consisting of a set of terms referring to one or more concepts, often arise in real-world querying scenarios such as; a complex search query into a document retrieval system or a nuanced subjective natural language question. The concepts in these queries may not directly refer to entities or canonical concept forms occurring in any fact-based or rule-based knowledge source such as a knowledge base or ontology. Thus, in addressing the complex information needs expressed by such novel contexts, systems using only such sources can fall short. Moreover, hidden associations meaningful in the current context, may not exist in a single document, but in a collection, between matching candidate concepts having different surface realizations, via alternate lexical forms. These may refer to underlying latent concepts, i.e., existing or conceived concepts or semantic classes that are accessible only via their surface forms. Inferring these latent concept associations in an implicit manner, by transferring knowledge from the same domain; within a collection, or from across domains (different collections), can potentially better address such novel contexts. Thus latent concept associations may act as a proxy for a novel context. This research hypothesizes that leveraging hidden associations between latent concepts may help to address novel contexts in a downstream recommendation task, and that knowledge transfer methods may aid and augment this process. With novel contexts and latent concept associations as the foundation, I define the process of concept discovery from text by two steps: first, "matching"; the novel context to an appropriate hidden relation between latent concepts, and second, "retrieving" the surface forms of the matched related concept as the discovered terms or concept.
Our prior study provides insight into how the transfer of knowledge within and across domains can help to learn associations between concepts, informing downstream prediction and recommendation tasks. In developing prediction models to explain factors affecting newspaper subscriber attrition or "churn", a set of "global" coarse-grained concepts or topics were learned on a corpus of web documents from a News domain , and later "inferred" on a parallel corpus of user search query logs belonging to a Clicklog domain. This process was then repeated in reverse and the topics learned on both domains were used in turn, as features into models predicting customer churn. The results in terms of the most predictive topics from the twin prediction tasks then allow us to reason about and understand how related factors across domains provide complementary signals to explain customer engagement.
This dissertation focuses on three main research contributions to improve seman- tic matching for downstream recommendation tasks via knowledge transfer. First, I employ a phrasal embedding-based generalized language model (GLM), to rank the other documents in a collection against each "query document" as a pseudo relevance feedback (PRF)-based scheme for generating semantic tag recommendations. This effectively leads to knowledge transfer "within" a domain by way of inferring related terms or fine-grained concepts for semantic tagging of documents, from existing documents in a collection. These semantic tags when used downstream in query expansion for information retrieval, both in direct and pseudo-relevance feedback query settings, give statistically significant improvement over baseline models that use word embedding-based or human expert-based query expansion terms. Next, inspired by the recent success of sequence-to-sequence neural models in delivering the state-of-the-art in a wide range of NLP tasks, I broaden the scope of the phrasal embedding-based generalized language model, to develop a novel end-end sequence-to-set framework (Seq2Set) with neural attention, for learning document representations for semantically tagging a large collection of documents with no previous labels, in an inductive transfer learning setting via self-taught learning. Seq2Set extends the use case of the previous GLM framework from an unsupervised PRF-based query expansion task set- ting, to supervised and semi-supervised task settings for automated text categorization via multi-label prediction. Using the Seq2Set framework we obtain statistically significant improvement over both; the previous phrasal GLM framework for the unsupervised query expansion task and also over the current state-of-the-art for the automated text categorization task both in the supervised as well as the semi-supervised settings.
The final contribution is to learn to answer complex, subjective, specific queries, given a source domain of "answered questions" about products that are labeled, by using a target domain of rich opinion data, i.e. "product reviews" that are unlabeled, by the novel application of neural domain adaptation in a transductive transfer learning setting. We learn to classify both labeled answers to questions as well as unlabeled review sentences via shared feature learning for appropriate knowledge transfer across the two domains, outperforming state-of-the-art baseline systems for sentence pair modeling tasks. Thus, given training labels on answer data, and by leveraging potential hidden associations between concepts in review and answer data, and reviews and query text, we are able to infer suitable answers from review text. We employ strategies such as maximum likelihood estimation-based neural generalized language modeling, sequence-to-set multi-label prediction with self-attention, and neural domain adaptation in these works. Combining these strategies with distributional semantics- based representations of surface forms of concepts, within neural frameworks that can facilitate knowledge transfer within and across domains, we demonstrate improved se- mantic matching, in downstream recommendation tasks, e.g. in finding related terms to address novel contexts in complex user queries, in a step towards really "finding what is meant" via concept discovery from text.