Driven Decision Making (IT728A_HT2019): Assignment 2
Transparency and Uncertainty Visualization
This individual assignment focuses on the process of prototyping a visual interface for a (real) data driven Decision Support System (DSS) flanking human decision makers. Section 1: Description of the tasks
- Identify a (real and teacher-accessible) data set of interest (examples of possible datasets and dataset sources can be found on the description page of Assignment 2 on Canvas).
- Formulate a decision problem that can be addressed by using the information contained in the dataset, and motivate its importance. Identify your assumptions, and the profile of the target user, i.e. the individual who, supported by your system, will actually be addressing the decision problem (e.g. domain experts, experts in machine learning techniques, novices, etc.). This specification will have a crucial impact on the motivation behind several of your next choices.
- Investigate and report sources of uncertainty affecting your dataset (e.g. accuracy, disagreement among multiple or heterogeneous sensors, incomplete data, inaccurate modeling, etc.).
- Describe the techniques applied to the analysis of your dataset and select the fundamental results that will inform a solution to your decision problem. Determine how a relevant subset of data (i.e. specific features in the dataset) can be selected to inform a solution to your decision problem and justify your choice based on your analysis and results.
- Organize the information on your visual interface for the DSS prototype, fine-tuning an appropriate trade-off between principles of transparency (explicitly including techniques for uncertainty visualization) and cognitive load on the target decision maker. Motivate your choices and their impact on enabling users’ deep insight on the decision problem, i.e. supporting situation assessment and decision making. At the same time modulate the effect of possible cognitive biases in the decision process. Any graphic tool for quick prototyping is allowed (e.g. Tableau, Spotfire, etc.).
- Describe and discuss your solution, analyzing its strong and weak points, also with reference to general principles of normative and descriptive decision theory.
Example: Using the “ship fuel consumption” data set (ref. task 1), we could ask how the data could inform expert vessel pilots’ decisions aiming to the reduction of fuel consumption during navigation in a real environment, leading to financial and environmental benefits (task 2). Working on the data set, we will soon discover that problems intrinsic to the sensors and with the physical context of navigation in open waters introduce several elements of uncertainty (3), and that some signals show a strong correlation with fuel consumption, thus proving more significant than others to address the question at hand (task 4). The relevant information will be displayed on the user interface, balancing quantity of information and representation of uncertainty within limits that the target decision maker can actually tolerate (task 5). Of course, the discussion (task 6) is crucially connected to the details of the actual implementation!
Section 2: Installments
Installment 1 (Oct 4 – facultative individual meeting, circa 30’/student)
Start populating your report template (available at the description page of Assignment 2 on Canvas) with a description of your dataset, decision problem and profile of the typical decision maker who will be targeted by your system. Tentatively, list the techniques that you are planning on using to analyze data and to visualize uncertainty, provide a sketch of your prototype describing your visual interface, and explain why it will be effective in the chosen domain. The goal of this installment is to formulate a tentative (and obviously partial) overview of the whole process, and to sketch a tentative draft visual interface before dwelling into further details of the solution. If possible, bring your own laptop to display the current state of the project and manuscript, otherwise email your up to date report shortly before the meeting.
Installment 2 (Oct 15 – facultative individual meeting, circa 30’/student)
Extend the material presented with Installment 1, focusing on its weakest points first. Use graphic tools or program a solution yourself to quickly produce results and motivate your choices. Produce sketches to try out your different interface design ideas. The purpose of this second installment is to demonstrate that the project is progressing along the right track. Perfection is not required, but the material will show the potential to quickly converge into a concrete solution. If possible, bring your own laptop to display the current state of the manuscript and project, otherwise email your up to date report shortly before the meeting.
Installment 3 (October 25, 2019 – mandatory oral presentation, circa 20’/student)
At this late stage of development, your project will address all of the points specified in Section 1 and you will be able to motivate your choices in front of an audience. Your presentation (circa 20 minutes including questions) will highlight the main elements that you have developed for your solution, by briefly introducing your demo of an interactive prototype, and your main findings. As scheduled, presentations will be delivered on October 25th (a list with individual time slots will be communicated before the event).
Installment 4 (due within November 4, 2019 – submission of the final report)
At this final stage of development, your report will address all of the points specified in Section 1, and will also integrate a reply to eventual constructive feedback received during the final oral presentation. Please upload a .pdf version of your report.
● Keep in mind that the focus of this assignment is on the design of a DSS transparent interface towards the human decision maker, and not on sophisticated technical solutions!
● Therefore, your prototype serves as “proof of concept”. It does not have to be perfect, and a minimalist demo of your prototype can be just fine. Rather, prioritize the work on motivating your design choices, particularly with reference to general principles transparency, and of normative and descriptive decision theory!
● Arranging an individual meeting is required for installments 1 and 2.
● At the moment of submission, check your manuscript for plagiarism.
● For clarifications regarding the assignment.