Algorithm-Driven Design: How Artificial Intelligence Is Changing Design

Personalizing UX Link

One way to get a clear and well-developed strategy is to personalize a product for a narrow audience segment or even specific users. We see it every day in Facebook newsfeeds, Google search results, Netflix and Spotify recommendations, and many other products. Besides the fact that it relieves the burden of filtering information from users, the users’ connection to the brand becomes more emotional when the product seems to care so much about them.

However, the key question here is about the role of designer in these solutions. We rarely have the skill to create algorithms like these — engineers and big data analysts are the ones to do it. Giles Colborne of CX Partners sees a great example in Spotify’s Discover Weekly feature: The only element of classic UX design here is the track list, whereas the distinctive work is done by a recommendation system that fills this design template with valuable music.

³⁴

Colborne offers advice to designers³⁵ about how to continue being useful in this new era and how to use various data sources to build and teach algorithms. It’s important to learn how to work with big data and to cluster it into actionable insights. For example, Airbnb learned how to answer the question, “What will the booked price of a listing be on any given day in the future?” so that its hosts could set competitive prices³⁶. There are also endless stories about Netflix’s recommendation engine.

I already mentioned some examples of automatic testing of design variations used by Netflix, Vox Media and The Grid. This is one more way to personalize UX that could be put onto the shoulders of algorithms. Liam Spradlin describes the interesting concept of mutative design³⁹; it’s a well-though-out model of adaptive interfaces that considers many variables to fit particular users.

An Exoskeleton For Designers Link

I’ve covered several examples of algorithm-driven design in practice. What tools do modern designers need for this? If we look back to the middle of the last century, computers were envisioned as a way to extend human capabilities. Roelof Pieters and Samim Winiger have analyzed computing history and the idea of augmentation of human ability⁴⁰ in detail. They see three levels of maturity for design tools:

1. First-generation systems mimic analogue tools with digital means.
2. The second generation is assisted creation systems, where humans and machines negotiate the creative process through tight action-feedback loops.
3. The third generation is assisted creation systems 3.0, which negotiate the creative process in fine-grained conversations, augment creative capabilities and accelerate the acquisition of skills from novice to expert.

⁴¹

Algorithm-driven design should be something like an exoskeleton for product designers — increasing the number and depth of decisions we can get through. How might designers and computers collaborate?

The working process of digital product designers could potentially look like this:

1. Explore a problem space, and pick the most valuable problem for the business and users to solve (analysis).
2. Explore a solution space, and pick the best solution to fix the problem (analysis).
3. Develop, launch and market a product that solves this problem (synthesis).
4. Evaluate how the product works for real users, and optimize it (analysis and synthesis).
5. Connect and unify the solution with other products and solutions of the company (synthesis).

These tasks are of two types: the analysis of implicitly expressed information and already working solutions, and the synthesis of requirements and solutions for them. Which tools and working methods do we need for each of them?

Analysis Link

Analysis of implicitly expressed information about users that can be studied with qualitative research is hard to automate. However, exploring the usage patterns of users of existing products is a suitable task. We could extract behavioral patterns and audience segments, and then optimize the UX for them. It’s already happening in ad targeting, where algorithms can cluster a user using implicit and explicit behavior patterns (within either a particular product or an ad network).

To train algorithms to optimize interfaces and content for these user clusters, designers should look into machine learning⁴³. Jon Bruner gives⁴⁴ a good example: A genetic algorithm starts with a fundamental description of the desired outcome — say, an airline’s timetable that is optimized for fuel savings and passenger convenience. It adds in the various constraints: the number of planes the airline owns, the airports it operates in, and the number of seats on each plane. It loads what you might think of as independent variables: details on thousands of flights from an existing timetable, or perhaps randomly generated dummy information. Over thousands, millions or billions of iterations, the timetable gradually improves to become more efficient and more convenient. The algorithm also gains an understanding of how each element of the timetable — the take-off time of Flight 37 from O’Hare, for instance — affects the dependent variables of fuel efficiency and passenger convenience.

In this scenario, humans curate an algorithm and can add or remove limitations and variables. The results can be tested and refined with experiments on real users. With a constant feedback loop, the algorithm improves the UX, too. Although the complexity of this work suggests that analysts will be doing it, designers should be aware of the basic principles of machine learning. O’Reilly published⁴⁵ a great mini-book on the topic recently.

Synthesis Link

Two years ago, a tool for industrial designers named Autodesk Dreamcatcher⁴⁶ made a lot of noise and prompted several publications from UX gurus⁴⁷. It’s based on the idea of generative design, which has been used in performance, industrial design, fashion and architecture for many years now. Many of you know Zaha Hadid Architects; its office calls this approach “parametric design⁴⁸.”

⁴⁹

Logojoy⁵¹ is a product to replace freelancers for a simple logo design. You choose favorite styles, pick a color and voila, Logojoy generates endless ideas. You can refine a particular logo, see an example of a corporate style based on it, and order a branding package with business cards, envelopes, etc. It’s the perfect example of an algorithm-driven design tool in the real world! Dawson Whitfield, the founder, described machine learning principles behind it⁵².

⁵³

However, it’s not yet established in digital product design, because it doesn’t help to solve utilitarian tasks. Of course, the work of architects and industrial designers has enough limitations and specificities of its own, but user interfaces aren’t static — their usage patterns, content and features change over time, often many times. However, if we consider the overall generative process — a designer defines rules, which are used by an algorithm to create the final object — there’s a lot of inspiration. The working process of digital product designers could potentially look like this:

1. An algorithm generates many variations of a design using predefined rules and patterns.
2. The results are filtered based on design quality and task requirements.
3. Designers and managers choose the most interesting and adequate variations, polishing them if needed.
4. A design system runs A/B tests for one or several variations, and then humans choose the most effective of them.

It’s yet unknown how can we filter a huge number of concepts in digital product design, in which usage scenarios are so varied. If algorithms could also help to filter generated objects, our job would be even more productive and creative. However, as product designers, we use generative design every day in brainstorming sessions where we propose dozens of ideas, or when we iterate on screen mockups and prototypes. Why can’t we offload a part of these activities to algorithms?

Experiment by Jon Gold Link

The experimental tool Rene⁵⁵ by Jon Gold, who worked at The Grid, is an example of this approach in action. Gold taught a computer to make meaningful typographic decisions⁵⁶. Gold thinks that it’s not far from how human designers are taught, so he broke this learning process into several steps:

1. Analyze glyphs to understand similarities in typefaces.
2. Formulate basic guidelines for combining typefaces.
3. Identify the best examples of type combinations to understand trends.
4. Create algorithms to observe how great designers work.

His idea is similar to what Roelof and Samim say: Tools should be creative partners for designers, not just dumb executants.

⁵⁷

Gold’s experimental tool Rene is built on these principles⁵⁸. He also talks about imperative and declarative approaches to programming and says that modern design tools should choose the latter — focusing on what we want to calculate, not how. Jon uses vivid formulas to show how this applies to design and has already made a couple of low-level demos. You can try out the tool⁵⁹ for yourself. It’s a very early concept but enough to give you the idea.