Interpretive Temporal Sequence Visualization

Many real-world applications produce event sequence data containing information crucial to important decision making. The ability to discover patterns of behavior or operational significance from multivariate temporal sequences (MTS) is essential to tasks such as purchase behavior analysis, sales performance management, manufacturing operation analysis, and electronic health record analysis. In practice, analyzing MTS is universally challenging due to the complexity in both the temporal and the feature dimensions. Further challenges are introduced by both the application-specific analysis requirements and the distinctive data patterns of interest. This dissertation addresses multivariate temporal sequence analysis with a focus on visualization assisted, objective-driven feature revelation and key pattern interpretation. The studies in this dissertation explore from descriptive to prescriptive approaches. Descriptive visual analytics focuses on providing data insights by visually summarizing and analyzing different aspects of data using data aggregation and data mining. Prescriptive visual analytics, based on optimization and prediction, focuses on learning insights from historical or real-time data using machine learning and statistical techniques and visually providing evidence that helps achieve the best future outcome. In this dissertation, I first develop a descriptive visual analysis system that automatically extracts temporal variations of cellular features from microscopy videos, and visualizes multiperspective cellular behavior in a spreadsheet-like layout. Second, I develop an interactive visualization system that lets domain analysts explore MTS in a prescriptive manner. Through intuitive visual exploration, sales performance analysts can reveal the relationship between sales behaviors and sales outcomes using linked views. The discovered key factors and temporal patterns help make further sales management decisions. Third, I train a recurrent neural network (RNN) model and develop a visual analytics system that extracts temporal events contributing to sequence predictions using the RNN attention mechanism. This approach helps domain specialists discover underlying temporal patterns that contribute to customer churn from massive tanglesome customer interaction events. This method also helps to improve the reliability of machine intelligence by letting users understand the features and values that are important to maintaining a healthy customer relationship. Fourth, I improve prior prescriptive approaches by developing a visual summary algorithm for contributing temporal pattern visualization. This algorithm incorporates agglomerative clustering to summarize temporal patterns in each prediction class. Then the algorithm distills temporal patterns by reducing noise generated by the machine learning model. The approach is evaluated in two application scenarios: electronic health records (EHR) analysis and open university learning analytics dataset (OULAD) analysis. The case study results show that the approach combining descriptive and prescriptive analyses helps users quickly identify and compare major contributing temporal patterns in different prediction classes. Later, I train a bidirectional RNN model that outperforms the previous RNN model and redesign the visualization of the temporal pattern summary system. The improved system enables an end-user-oriented analysis experience where the backend algorithms work in an encapsulated manner. The case studies are conducted using both RNN models for the EHR and OULAD analyses. The study results demonstrate that the temporal pattern summary approach can be generally used for different sequence prediction models. Finally, I develop a visual analytics tool for interactive what-if analysis. The tool lets users perturb sequence data and visualizes the inference results with a pre-trained RNN model in real-time. Globally, I design a partial dependence plot for clustering and summarizing the prediction responses to different feature values across the prediction classes. On the individual instance level, I calculate counterfactuals for user-selected instances. The partial dependence and counterfactual investigations augment the prescriptive analysis by providing descriptive evidence that guides users in making and testing hypotheses towards the desired outcome.