Facial Recording

Naturalistic assessment of language and emotion

FNIBI / Current Projects, Grants / Aphasia, Current Projects, Diffusion MRI, Eye-tracking, Facial Recording, Functional MRI, Language Task, Neuroimaging, Participate in a Study, Presurgical Planning /

September 26, 2023

Naturalistic paradigms hold enormous potential for studying abnormal brain function. However, the development of clinical applications based on naturalistic approaches is still in its infancy. Our symposium at OHBM 2023 presented the challenges and recent advances in research directed at clinical translation of naturalistic paradigms for brain mapping. See here if you are interested in participating.

The project titled “Naturalistic Neuroimaging for Presurgical Language Mapping” funded by the NIH/NIDCD, aims to evaluate movie fMRI language mapping in neurosurgical patients with varying levels of language function. As part of this project, we evaluate the comprehension and expression elicited by movie-watching, versus conventional language mapping paradigms (covert sentence completion, covert antonym generation, passive story listening), in individuals with stroke-induced aphasia and healthy controls. Language performance is assessed after each paradigm with explicit language tasks based on the stimulus material presented in the paradigm, as well as with implicit measures. We predict that language performance will be comparable, and in most individuals with aphasia superior, for movie-watching relative to the other paradigms. See, our pilot data for this project in this paper.

DynAMos: “The Dynamic Affective Movie Clip Database for Subjectivity Analysis” is a new video database that includes holistic and dynamic emotion ratings from over 100 participants watching 22 affective movie clips. Recognizing the subjectivity inherent to emotional experiences, we provide in the database the full distribution of all participants’ ratings. We argue that this choice represents a paradigm shift with the potential to unlock new research directions, generate new hypotheses, and inspire novel methods in the Affective Computing community. There are several interdisciplinary use cases for the database: to provide dynamic norms for emotion elicitation studies (e.g., in psychology, medicine, and neuroscience), to train and test affective content analysis algorithms (e.g., for dynamic emotion recognition, video summarization, and movie recommendation), and to study subjectivity in emotional reactions (e.g., to identify moments of emotional ambiguity or ambivalence within movies, identify predictors of subjectivity, and develop personalized affective content analysis algorithms). The database is made freely available to researchers for noncommercial use at https://dynamos.mgb.org.

An Ethics Checklist for Digital Health Research in Psychiatry: Viewpoint

FNIBI / Featured publications / Actigraphy, Digital Health, Digital phenotyping, Ethics, Facial Recording, Neuroimaging, Smartphone /

February 1, 2022

Psychiatry has long needed a better and more scalable way to capture the dynamics of behavior and its disturbances, quantitatively across multiple data channels, at high temporal resolution in real time. By combining 24/7 data—on location, movement, email and text communications, and social media—with brain scans, genetics, genomics, neuropsychological batteries, and clinical interviews, researchers will have an unprecedented amount of objective, individual-level data. Analyzing these data with ever-evolving artificial intelligence could one day include bringing interventions to patients where they are in the real world in a convenient, efficient, effective, and timely way. Yet, the road to this innovative future is fraught with ethical dilemmas as well as ethical, legal, and social implications (ELSI).

For more information:

https://www.jmir.org/2022/2/e31146/

The goal of the Ethics Checklist is to promote careful design and execution of research. It is not meant to mandate particular research designs; indeed, at this early stage and without consensus guidance, there are a range of reasonable choices researchers may make. However, the checklist is meant to make those ethical choices explicit, and to require researchers to give reasons for their decisions related to ELSI issues. The Ethics Checklist is primarily focused on procedural safeguards, such as consulting with experts outside the research group and documenting standard operating procedures for clearly actionable data (eg, expressed suicidality) within written research protocols.

We explored the ELSI of digital health research in psychiatry, with a particular focus on what we label “deep phenotyping” psychiatric research, which combines the potential for virtually boundless data collection and increasingly sophisticated techniques to analyze those data. We convened an interdisciplinary expert stakeholder workshop in May 2020, and this checklist emerges out of that dialogue.

Consistent with recent ELSI analyses, we find that existing ethical guidance and legal regulations are not sufficient for deep phenotyping research in psychiatry. At present, there are regulatory gaps, inconsistencies across research teams in ethics protocols, and a lack of consensus among institutional review boards on when and how deep phenotyping research should proceed. We thus developed a new instrument, an Ethics Checklist for Digital Health Research in Psychiatry (“the Ethics Checklist”). The Ethics Checklist is composed of 20 key questions, subdivided into 6 interrelated domains: (1) informed consent; (2) equity, diversity, and access; (3) privacy and partnerships; (4) regulation and law; (5) return of results; and (6) duty to warn and duty to report.

Deep phenotyping research offers a vision for vastly more effective care for people with, or at risk for, psychiatric disease. The potential perils en route to realizing this vision are significant; however, and researchers must be willing to address the questions in the Ethics Checklist before embarking on each leg of the journey.

Context-Adaptive Multimodal Informatics for Psychiatric Discharge Planning

FNIBI / Current Projects / Actigraphy, Audio Diary Recording, Bipolar Disorder, Current Projects, Depression, Discharge Planning, Dyadic Recording, Ecological Momentary Assessment, Electronic Health Record, Facial Recording, Inpatient, Longitudinal, Road to Recovery, Schizophrenia, Smartphone, Social Signal Processing Algorithms /

January 1, 2022

Determining when a patient is ready for discharge is an open problem, as currently 20% of those hospitalized for psychiatric illness will be hospitalized again within 30 days. To address this issue, we are working to identify behavioral biomarkers for severe mental illness in 400 inpatients hospitalized for psychosis at McLean. The Multisense project is a collaboration with Dr. with LP Morency‘s team at Carnegie Mellon University, where information is extracted from both audio and video of both clinician and patient during a series of semi-structured inpatient interviews. Multimodal analysis techniques are used both to predict clinical scale and discharge-readiness scores, as well as to visualize a summary of each interview and extract relevant interpretable features.

Computational Analysis of Spoken Language in Acute Psychosis and Mania

FNIBI / Featured publications / Bipolar Disorder, Facial Recording, Inpatient, Schizophrenia /

February 11, 2021

Objectives:

This study aimed to (1) determine the feasibility of collecting behavioral data from participants hospitalized with acute psychosis and (2) begin to evaluate the clinical information that can be computationally derived from such data.

Methods:

Behavioral data was collected across 99 sessions from 38 participants

recruited from an inpatient psychiatric unit. Each session started with a semi-structured interview modeled on a typical “clinical rounds” encounter and included administration of the Positive and Negative Syndrome

Scale (PANSS).

For more information:

https://osf.io/preprints/psyarxiv/zc47p

Neurogenetic analysis in unipolar and bipolar depression

FNIBI / Current Projects / Bipolar Disorder, Current Projects, Depression, Facial Recording, fMRI, Neurocognitive Tasks, Neuroimaging, Schizophrenia, SCID /

December 1, 2019

This is a neuroimaging study conducted in collaboration with Avram Holmes’ lab at Yale University focusing on neurogenetic factors related to unipolar and bipolar depression.

Circuit dynamics underlying longitudinal fluctuations in mood and cognition in bipolar patients

FNIBI / Current Projects / A Year in the Life, Actigraphy, Arterial Spin Labeling, Audio Diary Recording, Bipolar Disorder, Completed Projects, Depression, Dyadic Recording, Ecological Momentary Assessment, Eye-tracking, Facial Recording, Functional MRI, Neurocognitive Tasks, Neuroimaging, Schizophrenia, Smartphone, Social Signal Processing Algorithms /

January 1, 2018

Research on affective disorders, such as bipolar disorder, and on psychotic disorders is hampered by a lack of basic understanding of the course of dynamic circuit properties that might underlie fluctuations in mood and cognition.Bipolar and psychotic disorders at its core are unstable clinical conditions: at its extremes, it can result in periods of profound changes in mood and cognition (i.e., mania, major depression, and psychosis). And yet, remarkably little has been done to understand the basic mechanisms underlying the fluctuating course common to these individuals. We hope to better understand and characterize the natural course of changes in mood and cognition and associated environmental variables in individuals with severe affective and psychotic disorders using mobile behavioral technology. We believe that this will further enable advances in our understanding of how these disorders develop, paving the way for the development and evaluation of new treatment strategies.

Intensive Real-Time Computational Phenotyping in Individuals with Borderline Personality Disorder

FNIBI / Past Projects / Actigraphy, Borderline Personality Disorder, Completed Projects, Dyadic Recording, Facial Recording, Smartphone, Social Signal Processing Algorithms /

January 1, 2010

The scientific motivation for longitudinal study of individuals with mental illness has increasingly become an imperative, if we are to translate biological insights into discoveries that benefit individual patients. For studies intended to reveal differences between patients and healthy individuals at a group level, characterizing patients during periods of profound changes in mental status is a sensible approach, since it maximizes biological signal while minimizing the need to keep track of patients over time.