Direct Manipulation

Direct manipulation can simply be thought of as a button displayed on a screen and a user clicking that button to perform some type of action. There is more to direct manipulation, but that is its essence. Shneiderman describes the goal of direct manipulation interfaces, which is to provide an interface that is predictable, where the user is in control, where information being displayed is comprehensible, and where the user knows they are responsible for the actions they take. Further, Shniederman describes the current state of direct manipulation interfaces as ones that allow for visual methods of “searching” through databases of information. Shneiderman (1997) describes the benefit of direct manipulation interfaces being that results of actions taken are directly visible, which because of this, Shneiderman thinks direct manipulation are key candidates for future advanced interfaces. Essentially, the idea is that direct manipulation would give users peace-of-mind that their actions are accountable and the consequenting outputs of their actions are visible.

Twenty years later, the defining qualities of direct manipulation interfaces largely remain the same. However, research in the area is not as prosperous as it seems to have been in the 1980’s and 1990’s. This could be due to the adoption of direct manipulation on such a wide scale, that research on the idea and a debate for “direct manipulation” itself is irrelevant. Notable research in the area of direct manipulation is done by Shneiderman (1982),  Shneiderman (1993), Hutchins et al. (1985), and Frohlich (1993). This research mostly focuses on what the defining qualities of direct manipulation are.

With direct manipulation, as shown in Shneiderman's FilmFinder, one of the present issues seems to be an overabundance of information. The FilmFinder displays thousands of films on a single page. This type of display would likely not be useful to someone who simply wants to find a movie relevant to them. It would be very difficult to cover all of this information in a short period of time. 

Shneiderman notes a problem with interface agents being that it would be difficult for a tool, such as an agent, to recognize or “pick up” all the possibilities that the human eye would see. Although this is an easy perspective to take, it is better to anticipate innovation and the possibilities of what a computer can do. Paap and Katz (2004) did research into disruptive innovation within organizations and found that organizations are best positioned when they focus on both disruption and sustainment. In the case of this debate, direct manipulation would be sustainment and interface agents would be disruption. Shneiderman, at this point, fails to take the viewpoint of disruptive innovation and looks to the familiarity of direct manipulation. Because of this, Shneiderman does not pursue the idea of how this simple human performed task could be performed by an agent, and the chance for disruptive innovation goes down the drain.

Interface Agents

Maes describes an interface agent as something that brings personalization to a system. This personalization would come in the forms of understanding “habits, preferences, and interests” of the user. This information collected on habits would then be used to perform actions for the user in the background. The user would not have to be present at the system, and the agent would perform actions based upon its understanding of you.

In today's technology landscape, interface agents are readily present. Many systems or products feature some type of agent. These agents perform work on behalf of the users. Typically, this work is on such a scale, that not even a single individual would be able to perform it on their own. The work that these agents do is also often coming from an expert level, so if a user would want to do the work rather than the agent, they would either need to be an expert in the area or consult with an expert. One popular use of agents is the Netflix browse page, which is the homepage of its logged-in website. Netflix uses agents in that it recommends movies to people based on a large number of things it knows about them: past movies watched, past TV shows watched, length of time before abandoning, etc. With everything that Netflix knows about a specific user, it provides recommendations. These recommendations are always changing, and are meant to give the user an option that they would enjoy. This type of system would likely be impossible with direct manipulation, or it would require extreme expertise to continuously determine ideal movies.

Maes takes the position that direct manipulation is still needed in the wake of interface agents. She takes the opposite perspective of Sheiderman, and in doing so points out that if there is to be disruptive innovation, such as interface agents, direct manipulation needs to be sustained.

Key Points

Shneiderman argues that “responsibility” is the central issue in the debate. He thinks that direct manipulation allows for ultimate responsibility to be put onto the user. If a user performs an expected action, they will know the expected outcome. Further, he believes that direct manipulation reduces error and encourages exploration. Maes takes the approach that the system should offload work that is too time consuming to an agent. This is a simple approach, and one that would personalize certain actions to the user. Maes does not argue against direct manipulation, but notes how a beautifully designed interface would compliment an interface agent.

Thoughts on Direct Manipulation versus Interface Agents

From my perspective, it is very difficult to argue purely for direct manipulation or see a future where this would be the ultimate method to interface with a computer. It could be easy to take the side of direct manipulation due to placing all responsibility on the user and having confidence in less user errors. But to counter this argument by Shneiderman, there are many safeguards, through software testing, that could be put into place to ensure an interface with agents can still perform responsible and reliable actions.

In the world of software and systems development, increasing efficiency is often pursued by many teams from human factors, software, business, etc. If your team can increase the efficiency of your product, you and your product or system will likely be looked upon positively. One way to increase efficiency is through the use of agents, an example of this is described in the next paragraph.

If a system has a calculator, for example, that requires a user to enter several numbers and perform a mathematical operation, that could be very time consuming. The user would know exactly what they entered, but they would need mathematical expertise to ensure a correct answer. They are “responsible” for what they entered as Shneiderman would argue being his central benefit to direct manipulation, but the outcome is likely far from correct, as Shneiderman would have hoped for. To make this calculator more efficient, the designers of the system could employ agents to take the numbers and automatically perform the calculations without users knowing what is happening behind the scenes. The software could be tested to ensure it is extremely reliable, and direct manipulation would no longer be needed. Thus, an agent was employed, efficiency was created, and the use of a pure direct manipulation interface was terminated in the process.

Combining Both Styles of Interaction

I agree with Maes perspective that the two types of interfaces should be combined. There should always be some sort of direct manipulation, but interface agents can be employed to do tedious work and be used to personalize an experience and accomplish tasks that would be far too difficult for a human user.

Social & Legal Implications of Agents

One problem with agents, is that the user is not always in control. They may perform an action in the system, but what happens behind the scenes can be a mystery. This could be problematic in the case of poor design or designers being misled about what an agent does, which Shneiderman thinks would be one of the biggest problems with agents. Designers and other stakeholders do have a responsibility, and this area is extremely controversial now (Bessi and Ferrara, 2016) and could be an interesting field to explore in more depth.

References

Bessi, A., & Ferrara, E. (2016). Social bots distort the 2016 US Presidential election online discussion. First Monday, 21(11-7).

Frohlich, D. M. (1993). The history and future of direct manipulation. Behaviour & Information Technology, 12(6), 315-329.

Hutchins, E. L., Hollan, J. D., & Norman, D. A. (1985). Direct manipulation interfaces. Human-computer interaction, 1(4), 311-338.

Maes, P., Shneiderman, B., & Miller, J. (1997). Intelligent Software Agents vs. User-Controlled Direct Manipulation: A Debate, Panel Description. In Extended Abstracts of ACM CHI (Vol. 97).

Paap, J., & Katz, R. (2004). Anticipating disruptive innovation. Research-Technology Management, 47(5), 13-22.

Shneiderman, B. (1982). The future of interactive systems and the emergence of direct manipulation. Behaviour & Information Technology, 1(3), 237-256.

Shneiderman, B. (1993). 1.1 direct manipulation: a step beyond programming languages. Sparks of innovation in human-computer interaction, 17, 1993.

Shneiderman, B. (1997, January). Direct manipulation for comprehensible, predictable and controllable user interfaces. In IUI (Vol. 97, pp. 33-39).

Shneiderman, B., & Maes, P. (1997). Direct manipulation vs. interface agents. interactions, 4(6), 42-61.