Conversations between patients and providers in clinical settings provide a source of natural language data that may reflect and correlate with the patients’ experience and response to the treatment they are receiving. When analyzing utterances in such conversations, it is not sufficient to consider each sentence in isolation, since its context may play a role in determining its semantic meaning. Recently, contextual information in natural language documents has been modeled using various techniques, such as recurrent neural networks with latent variables, or neural networks with attention mechanisms. In this paper, we present UnsuPerviSed conText AuGmEntation (Upstage), a classification framework that relies on both local and global contextual information from different sources. Upstage uses transformer models with pretrained language models and joint sentence representation to solve the task of classifying health topics in patient-provider conversations. In addition, Upstage leverages unlabeled corpora for pretraining and data augmentation to provide additional context, which leads to improved classification performance.

D. Min

Verónica Pérez-Rosas

S. Kuo

W. Herman

Rada Mihalcea

UPSTAGE: Unsupervised Context Augmentation for Utterance Classification in Patient-Provider Communication

Modern electronic health records (EHRs) provide data to answer clinically meaningful questions. The growing data in EHRs makes healthcare ripe for the use of machine learning. However, learning in a clinical setting presents unique challenges that complicate the use of common machine learning methodologies. For example, diseases in EHRs are poorly labeled, conditions can encompass multiple underlying endotypes, and healthy individuals are underrepresented. This article serves as a primer to illuminate these challenges and highlights opportunities for members of the machine learning community to contribute to healthcare.

M. Ghassemi

Tristan Naumann

Peter F. Schulam

A. Beam

I. Chen

R. Ranganath

A Review of Challenges and Opportunities in Machine Learning for Health.

The use of machine learning (ML) in healthcare raises numerous ethical concerns, especially as models can amplify existing health inequities. Here, we outline ethical considerations for equitable ML in the advancement of healthcare. Specifically, we frame ethics of ML in healthcare through the lens of social justice. We describe ongoing efforts and outline challenges in a proposed pipeline of ethical ML in health, ranging from problem selection to postdeployment considerations. We close by summarizing recommendations to address these challenges.

E. Pierson

Sherri Rose

Shalmali Joshi

Kadija Ferryman

Ethical Machine Learning in Health Care

Intimate partner violence (IPV) is an urgent, prevalent, and under-detected public health issue. We present machine learning models to assess patients for IPV and injury. We train the predictive algorithms on radiology reports with 1) IPV labels based on entry to a violence prevention program and 2) injury labels provided by emergency radiology fellowship-trained physicians. Our dataset includes 34,642 radiology reports and 1479 patients of IPV victims and control patients. Our best model predicts IPV a median of 3.08 years before violence prevention program entry with a sensitivity of 64% and a specificity of 95%. We conduct error analysis to determine for which patients our model has especially high or low performance and discuss next steps for a deployed clinical risk model.

Irene Chen

Emily Alsentzer

Hyesun Park

Richard Thomas

Babina Gosangi

Rahul Gujrathi

B. Khurana

Intimate Partner Violence and Injury Prediction From Radiology Reports

Objective
Patient notes in electronic health records (EHRs) may contain critical information for medical investigations. However, the vast majority of medical investigators can only access de-identified notes, in order to protect the confidentiality of patients. In the United States, the Health Insurance Portability and Accountability Act (HIPAA) defines 18 types of protected health information that needs to be removed to de-identify patient notes. Manual de-identification is impractical given the size of electronic health record databases, the limited number of researchers with access to non-de-identified notes, and the frequent mistakes of human annotators. A reliable automated de-identification system would consequently be of high value.


Materials and Methods
We introduce the first de-identification system based on artificial neural networks (ANNs), which requires no handcrafted features or rules, unlike existing systems. We compare the performance of the system with state-of-the-art systems on two datasets: the i2b2 2014 de-identification challenge dataset, which is the largest publicly available de-identification dataset, and the MIMIC de-identification dataset, which we assembled and is twice as large as the i2b2 2014 dataset.


Results
Our ANN model outperforms the state-of-the-art systems. It yields an F1-score of 97.85 on the i2b2 2014 dataset, with a recall of 97.38 and a precision of 98.32, and an F1-score of 99.23 on the MIMIC de-identification dataset, with a recall of 99.25 and a precision of 99.21.


Conclusion
Our findings support the use of ANNs for de-identification of patient notes, as they show better performance than previously published systems while requiring no manual feature engineering.

Franck Dernoncourt

J. Y. Lee

Özlem Uzuner

Peter Szolovits

De-identification of patient notes with recurrent neural networks

We propose Segment Convolutional Neural Networks (Seg-CNNs) for classifying relations from clinical notes. Seg-CNNs use only word-embedding features without manual feature engineering. Unlike typical CNN models, relations between 2 concepts are identified by simultaneously learning separate representations for text segments in a sentence: preceding, concept1, middle, concept2, and succeeding. We evaluate Seg-CNN on the i2b2/VA relation classification challenge dataset. We show that Seg-CNN achieves a state-of-the-art micro-average F-measure of 0.742 for overall evaluation, 0.686 for classifying medical problem-treatment relations, 0.820 for medical problem-test relations, and 0.702 for medical problem-medical problem relations. We demonstrate the benefits of learning segment-level representations. We show that medical domain word embeddings help improve relation classification. Seg-CNNs can be trained quickly for the i2b2/VA dataset on a graphics processing unit (GPU) platform. These results support the use of CNNs computed over segments of text for classifying medical relations, as they show state-of-the-art performance while requiring no manual feature engineering.

Yuan Luo

Yu Cheng

J. Starren

Segment convolutional neural networks (Seg-CNNs) for classifying relations in clinical notes

We present a system that uses a learned autocompletion mechanism to facilitate rapid creation of semi-structured clinical documentation. We dynamically suggest relevant clinical concepts as a doctor drafts a note by leveraging features from both unstructured and structured medical data. By constraining our architecture to shallow neural networks, we are able to make these suggestions in real time. Furthermore, as our algorithm is used to write a note, we can automatically annotate the documentation with clean labels of clinical concepts drawn from medical vocabularies, making notes more structured and readable for physicians, patients, and future algorithms. To our knowledge, this system is the only machine learning-based documentation utility for clinical notes deployed in a live hospital setting, and it reduces keystroke burden of clinical concepts by 67% in real environments.

Divya Gopinath

Monica Agrawal

Luke S. Murray

S. Horng

D. Karger

D. Sontag

Fast, Structured Clinical Documentation via Contextual Autocomplete

The COVID-19 Open Research Dataset (CORD-19) is a growing resource of scientific papers on COVID-19 and related historical coronavirus research. CORD-19 is designed to facilitate the development of text mining and information retrieval systems over its rich collection of metadata and structured full text papers. Since its release, CORD-19 has been downloaded over 200K times and has served as the basis of many COVID-19 text mining and discovery systems. In this article, we describe the mechanics of dataset construction, highlighting challenges and key design decisions, provide an overview of how CORD-19 has been used, and describe several shared tasks built around the dataset. We hope this resource will continue to bring together the computing community, biomedical experts, and policy makers in the search for effective treatments and management policies for COVID-19.

Lucy Lu Wang

Kyle Lo

Yoganand Chandrasekhar

Russell Reas

Jiangjiang Yang

Darrin Eide

Kathryn Funk

Rodney Michael Kinney

Ziyang Liu

William Cooper Merrill

P. Mooney

D. Murdick

Devvret Rishi

J. Sheehan

Zhihong Shen

Brandon Stilson

Alex D Wade

Kuansan Wang

Christopher Wilhelm

Boya Xie

Douglas A. Raymond

Daniel S. Weld

Oren Etzioni

Sebastian Kohlmeier

CORD-19: The COVID-19 Open Research Dataset

Background
As machine learning becomes increasingly common in health care applications, concerns have been raised about bias in these systems' data, algorithms, and recommendations. Simply put, as health care improves for some, it might not improve for all.


Methods
Two case studies are examined using a machine learning algorithm on unstructured clinical and psychiatric notes to predict intensive care unit (ICU) mortality and 30-day psychiatric readmission with respect to race, gender, and insurance payer type as a proxy for socioeconomic status.


Results
Clinical note topics and psychiatric note topics were heterogenous with respect to race, gender, and insurance payer type, which reflects known clinical findings. Differences in prediction accuracy and therefore machine bias are shown with respect to gender and insurance type for ICU mortality and with respect to insurance policy for psychiatric 30-day readmission.


Conclusions
This analysis can provide a framework for assessing and identifying disparate impacts of artificial intelligence in health care.

Can AI Help Reduce Disparities in General Medical and Mental Health Care?

In the United States, the country with the highest confirmed COVID-19 infection cases, a nationwide social distancing protocol has been implemented by the President. Following the closure of the University of Washington on March 7th, more than 1000 colleges and universities in the United States have cancelled in-person classes and campus activities, impacting millions of students. This paper aims to discover the social implications of this unprecedented disruption in our interactive society regarding both the general public and higher education populations by mining people's opinions on social media. We discover several topics embedded in a large number of COVID-19 tweets that represent the most central issues related to the pandemic, which are of great concerns for both college students and the general public. Moreover, we find significant differences between these two groups of Twitter users with respect to the sentiments they expressed towards the COVID-19 issues. To our best knowledge, this is the first social media-based study which focuses on the college student community's demographics and responses to prevalent social issues during a major crisis.

Viet-An Duong

Phu Pham

Tongyu Yang

Yu Wang

Jiebo Luo

The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic

Social networks are widely used for information consumption and dissemination, especially during time-critical events such as natural disasters. Despite its significantly large volume, social media content is often too noisy for direct use in any application. Therefore, it is important to filter, categorize, and concisely summarize the available content to facilitate effective consumption and decision-making. To address such issues automatic classification systems have been developed using supervised modeling approaches, thanks to the earlier efforts on creating labeled datasets. However, existing datasets are limited in different aspects (e.g., size, contains duplicates) and less suitable to support more advanced and data-hungry deep learning models. In this paper, we present a new large-scale dataset with ∼77K human-labeled tweets, sampled from a pool of ∼24 million tweets across 19 disaster events that happened between 2016 and 2019. Moreover, we propose a data collection and sampling pipeline, which is important for social media data sampling for human annotation. We report multiclass classification results using classic and deep learning (fastText and transformer) based models to set the ground for future studies. The dataset and associated resources are publicly available at https: //crisisnlp.qcri.org/humaid_dataset.html.

Firoj Alam

U. Qazi

Muhammad Imran

Ferda Ofli

HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks

During natural and man-made disasters, people use social media platforms such as Twitter to post textual and multime- dia content to report updates about injured or dead people, infrastructure damage, and missing or found people among other information types. Studies have revealed that this on- line information, if processed timely and effectively, is ex- tremely useful for humanitarian organizations to gain situational awareness and plan relief operations. In addition to the analysis of textual content, recent studies have shown that imagery content on social media can boost disaster response significantly. Despite extensive research that mainly focuses on textual content to extract useful information, limited work has focused on the use of imagery content or the combination of both content types. One of the reasons is the lack of labeled imagery data in this domain. Therefore, in this paper, we aim to tackle this limitation by releasing a large multi-modal dataset collected from Twitter during different natural disasters. We provide three types of annotations, which are useful to address a number of crisis response and management tasks for different humanitarian organizations.

CrisisMMD: Multimodal Twitter Datasets from Natural Disasters

The success of deep neural networks (DNNs) is heavily dependent on the availability of labeled data. However, obtaining labeled data is a big challenge in many real-world problems. In such scenarios, a DNN model can leverage labeled and unlabeled data from a related domain, but it has to deal with the shift in data distributions between the source and the target domains. In this paper, we study the problem of classifying social media posts during a crisis event (e.g., Earthquake). For that, we use labeled and unlabeled data from past similar events (e.g., Flood) and unlabeled data for the current event. We propose a novel model that performs adversarial learning based domain adaptation to deal with distribution drifts and graph based semi-supervised learning to leverage unlabeled data within a single unified deep learning framework. Our experiments with two real-world crisis datasets collected from Twitter demonstrate significant improvements over several baselines.

Shafiq R. Joty

Domain Adaptation with Adversarial Training and Graph Embeddings

We describe the Iraq Body Count Corpus (IBC-C) dataset, the first substantial armed conflict-related dataset which can be used for conflict analysis. IBC-C provides a ground-truth dataset for conflict specific named entity recognition, slot filling, and event de-duplication. IBC-C is constructed using data collected by the Iraq Body Count project which has been recording casualties resulting from the ongoing war in Iraq since 2003. We describe the dataset’s creation, how it can be used for the above three tasks and provide initial baseline results for the first task (named entity recognition) using Hidden Markov Models, Conditional Random Fields, and Recursive Neural Networks.

Andrej Žukov-Gregorič

Bartal Veyhe

Zhiyuan Luo

IBC-C : A Dataset for Armed Conflict Event Analysis

Computer-aided text analysis (CATA) offers exciting new possibilities for conflict research that this contribution describes using a range of exemplary studies from a variety of disciplines including sociology, political science, communication studies, and computer science. The chapter synthesizes empirical research that investigates conflict in relation to text across different formats and genres. This includes both conflict as it is verbalized in the news media, in political speeches, and other public documents and conflict as it occurs in online spaces (social media platforms, forums) and that is largely confined to such spaces (e.g., flaming and trolling). Particular emphasis is placed on research that aims to find commonalities between online and offline conflict, and that systematically investigates the dynamics of group behavior. Both work using inductive computational procedures, such as topic modeling, and supervised machine learning approaches are assessed, as are more traditional forms of content analysis, such as dictionaries. Finally, cross-validation is highlighted as a crucial step in CATA, in order to make the method as useful as possible to scholars interested in enlisting text mining for conflict research.

Seraphine F. Maerz

C. Puschmann

Text as Data for Conflict Research: A Literature Survey

We extend the well-known word analogy task to a one-to-X formulation, including one-to-none cases, when no correct answer exists. The task is cast as a relation discovery problem and applied to historical armed conflicts datasets, attempting to predict new relations of type ‘location:armed-group’ based on data about past events. As the source of semantic information, we use diachronic word embedding models trained on English news texts. A simple technique to improve diachronic performance in such task is demonstrated, using a threshold based on a function of cosine distance to decrease the number of false positives; this approach is shown to be beneficial on two different corpora. Finally, we publish a ready-to-use test set for one-to-X analogy evaluation on historical armed conflicts data.

Andrey Kutuzov

Erik Velldal

Lilja Øvrelid

One-to-X Analogical Reasoning on Word Embeddings: a Case for Diachronic Armed Conflict Prediction from News Texts

Current plagiarism detection tools are mostly limited to comparisons of suspicious plagiarised texts and potential original texts at string level. In this study the aim is to improve the accuracy of plagiarism detection by incorporating Natural Language Processing (NLP) techniques into existing approaches. We propose a framework for external plagiarism detection in which a number of NLP techniques are applied to process a set of suspicious and original documents, not only to analyse strings but also the structure of the text, using resources to account for text relations. Initial results obtained with a corpus of plagiarised short paragraphs have showed that NLP techniques improve the accuracy of existing approaches.

Miranda Chong

Lucia Specia

R. Mitkov

Using Natural Language Processing for Automatic Detection of Plagiarism

Traditional automated essay scoring systems rely on carefully designed features to evaluate and score essays. The performance of such systems is tightly bound to the quality of the underlying features. However, it is laborious to manually design the most informative features for such a system. In this paper, we develop an approach based on recurrent neural networks to learn the relation between an essay and its assigned score, without any feature engineering. We explore several neural network models for the task of automated essay scoring and perform some analysis to get some insights of the models. The results show that our best system, which is based on long short-term memory networks, outperforms a strong baseline by 5.6% in terms of quadratic weighted Kappa, without requiring any feature engineering.

K. Taghipour

H. Ng

A Neural Approach to Automated Essay Scoring

In this position paper, we argue that building operational automated scoring systems is a task that has disciplinary complexity above and beyond standard competitive shared tasks which usually involve applying the latest machine learning techniques to publicly available data in order to obtain the best accuracy. Automated scoring systems warrant significant cross-discipline collaboration of which natural language processing and machine learning are just two of many important components. Such systems have multiple stakeholders with different but valid perspectives that can often times be at odds with each other. Our position is that it is essential for us as NLP researchers to understand and incorporate these perspectives in our research and work towards a mutually satisfactory solution in order to build automated scoring systems that are accurate, fair, unbiased, and useful.

Nitin Madnani

A. Cahill

Automated Scoring: Beyond Natural Language Processing

This article presents the Event Data on Conflict and Security (EDACS) dataset, discusses the inherent problems of georeferenced conflict data, and shows how these challenges are met within EDACS. Based on an event data approach, EDACS contributes to the growing number of novel georeferenced datasets that allow researchers to identify causal pathways of violence and the dynamics of (transboundary) violence through spatiotemporal disaggregation. However, the unreflected use of any of these datasets will give researchers unjustified confidence in their findings, as the pitfalls are many and propagating errors can result in misleading conclusions. To identify and handle the different challenges to overall event data quality, we argue in favor of transparency in the data collection and coding process, to empower analysts to challenge the data and avoid cascading errors. In particular, we investigate how the choice of news sources, the handling of geographic precision, and the use of auxiliary data can bias event data. We demonstrate how the EDACS dataset design enables the analyst to deal with these issues by providing a set of variables indicating the news sources, possible sources of bias, and detailed information on geographic precision. This allows for a flexible use of the data based on individual analytical requirements.

S. Chojnacki

Christian Ickler

Michael Spies

J. Wiesel

Event Data on Armed Conflict and Security: New Perspectives, Old Challenges, and Some Solutions

Recent studies have shown that word embedding models can be used to trace time-related (diachronic) semantic shifts in particular words. In this paper, we evaluate some of these approaches on the new task of predicting the dynamics of global armed conflicts on a year-to-year basis, using a dataset from the conflict research field as the gold standard and the Gigaword news corpus as the training data. The results show that much work still remains in extracting ‘cultural’ semantic shifts from diachronic word embedding models. At the same time, we present a new task complete with an evaluation set and introduce the ‘anchor words’ method which outperforms previous approaches on this set.

Tracing armed conflicts with diachronic word embedding models

Textbooks play an important role in any educational system. Unfortunately, many textbooks produced in developing countries are not written well and they often lack adequate coverage of important concepts. We propose a technological solution to address this problem based on enriching textbooks with authoritative web content. We augment textbooks at the section level for key concepts discussed in the section. We use ideas from data mining for identifying the concepts that need augmentation as well as to determine the links to the authoritative content that should be used for augmentation. Our evaluation, employing textbooks from India, shows that we are able to enrich textbooks on different subjects and across different grades with high quality augmentations using automated techniques.

R. Agrawal

Sreenivas Gollapudi

K. Kenthapadi

Nitish Srivastava

R. Velu

Enriching textbooks through data mining

Question answering (QA) is the automated process of answering general questions submitted by humans in natural language. QA has previously been explored within the educational context to facilitate learning, however the majority of works have focused on text-based answering. As an alternative, this paper proposes an approach to return answers as a concept map, which further encourages meaningful learning and knowledge organisation. Additionally, this paper investigates whether adapting the returned concept map to the specific question context provides further learning benefit. A randomised experiment was conducted with a sample of 59 Computer Science undergraduates, obtaining statistically significant results on learning gain when students are provided with the question-specific concept maps. Further, time spent on studying the concept maps were positively correlated with the learning gain.

Thushari Atapattu

K. Falkner

Nickolas J. G. Falkner

Educational Question Answering Motivated by Question-Specific Concept Maps

Writing is a complex non-linear process that begins with a mental model of intent, and progresses through an outline of ideas, to words on paper (and their subsequent refinement). Despite past research in understanding writing, Web-scale consumer and enterprise collaborative digital writing environments are yet to greatly benefit from intelligent systems that understand the stages of document evolution, providing opportune assistance based on authors' situated actions and context. In this paper, we present three studies that explore temporal stages of document authoring. We first survey information workers at a large technology company about their writing habits and preferences, concluding that writers do in fact conceptually progress through several distinct phases while authoring documents. We also explore, qualitatively, how writing stages are linked to document lifespan. We supplement these qualitative findings with an analysis of the longitudinal user interaction logs of a popular digital writing platform over several million documents. Finally, as a first step towards facilitating an intelligent digital writing assistant, we conduct a preliminary investigation into the utility of user interaction log data for predicting the temporal stage of a document. Our results support the benefit of tools tailored to writing stages, identify primary tasks associated with these stages, and show that it is possible to predict stages from anonymous interaction logs. Together, these results argue for the benefit and feasibility of more tailored digital writing assistance.

Bahareh Sarrafzadeh

Sujay Kumar Jauhar

Michael Gamon

E. Lank

Ryen W. White

Characterizing Stage-aware Writing Assistance for Collaborative Document Authoring

Question Answering (QA) system in information retrieval is a task of automatically answering a correct answer to the questions asked by human in natural language using either a pre-structured database or a collection of natural language documents. It presents only the requested information instead of searching full documents like search engine. As information in day to day life is increasing, so to retrieve the exact fragment of information even for a simple query requires large and expensive resources. This is the paper which describes the different methodology and implementation details of question answering system for general language and also proposes the closed domain QA System for handling documents related to education acts sections to retrieve more precise answers using NLP techniques.

Sweta P. Lende

M. Raghuwanshi

Question answering system on education acts using NLP techniques

As a relatively young field of research and development started by work on cryptanalysis and machine translation around 50 years ago, Natural Language Processing (NLP) is concerned with the automated processing of human language. It addresses the analysis and generation of written and spoken language, though speech processing is often regarded as a separate subfield. NLP emphasizes processing and applications and as such can be seen as the applied side of Computational Linguistics, the interdisciplinary field of research concerned with formal analysis and modeling of language and its applications at the intersection of Linguistics, Computer Science, and Psychology. In terms of the language aspects dealt with in NLP, traditionally lexical, morphological and syntactic aspects of language were at the center of attention, but aspects of meaning, discourse, and the relation to the extra-linguistic context have become increasingly prominent in the last decade. A good introduction and overview of the field is provided in Jurafsky & Martin (2009).

C. Chapelle

Blackwell

Natural Language Processing and Language Learning

Management of collaborative documents can be difficult, given the profusion of edits and comments that multiple authors make during a document’s evolution. Reliably modeling the relationship between edits and comments is a crucial step towards helping the user keep track of a document in flux. A number of authoring tasks, such as categorizing and summarizing edits, detecting completed to-dos, and visually rearranging comments could benefit from such a contribution. Thus, in this paper we explore the relationship between comments and edits by defining two novel, related tasks: Comment Ranking and Edit Anchoring. We begin by collecting a dataset with more than half a million comment-edit pairs based on Wikipedia revision histories. We then propose a hierarchical multi-layer deep neural-network to model the relationship between edits and comments. Our architecture tackles both Comment Ranking and Edit Anchoring tasks by encoding specific edit actions such as additions and deletions, while also accounting for document context. In a number of evaluation settings, our experimental results show that our approach outperforms several strong baselines significantly. We are able to achieve a precision@1 of 71.0% and a precision@3 of 94.4% for Comment Ranking, while we achieve 74.4% accuracy on Edit Anchoring.

Xuchao Zhang

Dheeraj Rajagopal

Chang-Tien Lu

Modeling the Relationship between User Comments and Edits in Document Revision

A multimodal human-computer interaction system is composed of the comprehensive usage of various input and output channels. For the information input, apart from the traditional keyboard typing, mouse clicking, screen touching, the latest speech and face recognition technology can be used. For the output, the traditional screen display, the latest speech and facial expression synthesis and gesture generation can be used. After literature review of related works, this paper at first presents such a system, MMISE (Multimodal Interaction System for Education), about its architecture and working mechanism, POOOIIM (Pedagogical Objective Oriented Output, Input and Implementation Mechanism) illustrated with practical examples. Then this paper introduces this system’s pilot applications in the epidemic time of novel coronavirus in 2020.

Jiyou Jia

Yunfan He

Huixiao Le

A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments

The quality of a counseling intervention relies highly on the active collaboration between clients and counselors. In this paper, we explore several linguistic aspects of the collaboration process occurring during counseling conversations. Specifically, we address the differences between high-quality and low-quality counseling. Our approach examines participants’ turn-by-turn interaction, their linguistic alignment, the sentiment expressed by speakers during the conversation, as well as the different topics being discussed. Our results suggest important language differences in low- and high-quality counseling, which we further use to derive linguistic features able to capture the differences between the two groups. These features are then used to build automatic classifiers that can predict counseling quality with accuracies of up to 88%.

Xinyi Wu

K. Resnicow

What Makes a Good Counselor? Learning to Distinguish between High-quality and Low-quality Counseling Conversations

The COVID-19 pandemic, like many of the disease outbreaks that have preceded it, is likely to have a profound effect on mental health. Understanding its impact can inform strategies for mitigating negative consequences. In this work, we seek to better understand the effects of COVID-19 on mental health by examining discussions within mental health support communities on Reddit. First, we quantify the rate at which COVID-19 is discussed in each community, or subreddit, in order to understand levels of preoccupation with the pandemic. Next, we examine the volume of activity in order to determine whether the quantity of people seeking online mental health support has risen. Finally, we analyze how COVID-19 has influenced language use and topics of discussion within each subreddit.

Laura Biester

K. Matton

Janarthanan Rajendran

E. Provost

Quantifying the Effects of COVID-19 on Mental Health Support Forums

Data mining techniques have been applied to educational research in various ways. In this paper, I presented the application of sequential data mining algorithms to analyze computer logs to profile learners in terms of their learning tactic use and motivation in a web-based learning environment (gStudy). The data mining algorithms are employed to discover patterns which characterize learners either across session or groups based on their study tactic choice and goal orientation. The use of this method is illustrated through a sequential pattern analysis of gStudy log files generated by university students.

Mingming Zhou

Data Mining and Student e-Learning Profiles

Worldwide, an increasing number of people are suffering from mental health disorders such as depression and anxiety. In the United States alone, one in every four adults suffers from a mental health condition, which makes mental health a pressing concern. In this paper, we explore the use of multimodal cues present in social media posts to predict users’ mental health status. Specifically, we focus on identifying social media activity that either indicates a mental health condition or its onset. We collect posts from Flickr and apply a multimodal approach that consists of jointly analyzing language, visual, and metadata cues and their relation to mental health. We conduct several classification experiments aiming to discriminate between (1) healthy users and users affected by a mental health illness; and (2) healthy users and users prone to mental illness. Our experimental results indicate that using multiple modalities can improve the performance of this classification task as compared to the use of one modality at a time, and can provide important cues into a user’s mental status.

Z. Xu

Inferring Social Media Users’ Mental Health Status from Multimodal Information

The ongoing COVID-19 pandemic has raised concerns for many regarding personal and public health implications, financial security and economic stability. Alongside many other unprecedented challenges, there are increasing concerns over social isolation and mental health. We introduce \textit{Expressive Interviewing}--an interview-style conversational system that draws on ideas from motivational interviewing and expressive writing. Expressive Interviewing seeks to encourage users to express their thoughts and feelings through writing by asking them questions about how COVID-19 has impacted their lives. We present relevant aspects of the system's design and implementation as well as quantitative and qualitative analyses of user interactions with the system. In addition, we conduct a comparative evaluation with a general purpose dialogue system for mental health that shows our system potential in helping users to cope with COVID-19 issues.

Charles F Welch

Allison Lahnala

Siqi Shen

S. Seraj

Lawrence An

J. Pennebaker

Expressive Interviewing: A Conversational System for Coping with COVID-19

Counselor empathy is associated with better outcomes in psychology and behavioral counseling. In this paper, we explore several aspects pertaining to counseling interaction dynamics and their relation to counselor empathy during motivational interviewing encounters. Particularly, we analyze aspects such as participants’ engagement, participants’ verbal and nonverbal accommodation, as well as topics being discussed during the conversation, with the final goal of identifying linguistic and acoustic markers of counselor empathy. We also show how we can use these findings alongside other raw linguistic and acoustic features to build accurate counselor empathy classifiers with accuracies of up to 80%.

Satinder Singh

Understanding and Predicting Empathic Behavior in Counseling Therapy

Mental illness is one of the most pressing public health issues of our time. While counseling and psychotherapy can be effective treatments, our knowledge about how to conduct successful counseling conversations has been limited due to lack of large-scale data with labeled outcomes of the conversations. In this paper, we present a large-scale, quantitative study on the discourse of text-message-based counseling conversations. We develop a set of novel computational discourse analysis methods to measure how various linguistic aspects of conversations are correlated with conversation outcomes. Applying techniques such as sequence-based conversation models, language model comparisons, message clustering, and psycholinguistics-inspired word frequency analyses, we discover actionable conversation strategies that are associated with better conversation outcomes.

Tim Althoff

Kevin Clark

J. Leskovec

Large-scale Analysis of Counseling Conversations: An Application of Natural Language Processing to Mental Health

This paper describes our system (Fermi) for Task 6: OffensEval: Identifying and Categorizing Offensive Language in Social Media of SemEval-2019. We participated in all the three sub-tasks within Task 6. We evaluate multiple sentence embeddings in conjunction with various supervised machine learning algorithms and evaluate the performance of simple yet effective embedding-ML combination algorithms. Our team Fermi’s model achieved an F1-score of 64.40%, 62.00% and 62.60% for sub-task A, B and C respectively on the official leaderboard. Our model for sub-task C which uses pre-trained ELMo embeddings for transforming the input and uses SVM (RBF kernel) for training, scored third position on the official leaderboard. Through the paper we provide a detailed description of the approach, as well as the results obtained for the task.

Vijayasaradhi Indurthi

Fermi at SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media using Sentence Embeddings