We all have fears and doubts. It’s not different for you than for me. Over the last weeks, “well-known” people on Twitter started to share mistakes they made in life or their careers. I think it’s very helpful to read that we all make mistakes.
We all have to learn and improve, and people who are on a stage at an event for the 100th time are still known to be extremely nervous. Let’s realign our views, our expectations and, instead of being afraid of making mistakes, try to improve our knowledge and let others learn from the things that didn’t go as expected.
Caddy, an HTTP/2 server that has automatic HTTPS built in, was released in version 0.1010 and brings man-in-the-middle (MITM) attack detection and HTTP/2 Server Push.
secureoperator13 is a proxy for DNS that uses Google’s DNS over HTTPS14 technology. A nice experiment that brings security to a still weak bridge. And while technologies to add security to the DNS do already exist (DANE and DNSSEC, for example), they’re not as widespread and not free of weak points. However, using DNS via Google also means trusting a third party that could intercept the requests at any time. One thing is for certain, according to their privacy policy15, they do store logs with your IP address and other information.
Due to its improvements over MySQL and independence from Oracle, MariaDB is getting lots of traction at the moment. However, there are certain differences in how MariaDB/MySQL and PostgreSQL handle data16. If you take a closer look, you’ll notice that running into weird miscalculations or errors is much more likely with MariaDB/MySQL while PostgreSQL will return a strict fail if a value doesn’t match a field type.
Clémentine Maurice and other researchers found a way to steal data from the CPU cache shared by two Virtual Machines17. This was demonstrated on Amazon Web Services but affects all Virtual Machine-based environments. Clear evidence that we still have little idea of how secure or insecure cloud environments actually are.
Amazon announced “Echo Look”, an improved Alexa device that does not only listen to a room’s activity but also has a camera18 to see what’s happening. The purpose? To give you a style check. And as you would expect from Amazon, they say they store the captured data for an indefinite amount of time in their cloud. I bet that a lot of people will buy this device despite of this, even those who claim to care about their privacy.
Jonathan Taplin wrote an essay about the tech moguls dominating the free market today25 and why it’s important that we as consumers are aware of the huge influence monopolies have not only on our lives but on politics, too.
The outdoor clothing manufacturer Patagonia started to sell used clothing for little money26. An unusual move for such a company as it undermines its traditional business model of selling new clothes.
Iterating on their already existing, centralized AI technology, Google researchers shared their vision of federated machine learning27. This basically means that every Google device will contribute to the training data by locally processing the information — a much more efficient and less costly approach for Google. The technology is already being tested on Android via Google’s software keyboard. Let’s see how this will work out when it comes to dealing with fake news, spam content or violence promotion in Google’s search results.
Mastodon is a relatively new social microblogging network, aiming to replace Twitter. It uses a federated approach, which means everyone can create an instance that shares data with other instances. But it’s not as easy as one would initially think. By providing an instance, you suddenly become responsible for the content of other people, which can be a pretty nasty experience as this story shows28.
I’ve been a long time Photoshop and Illustrator user. Both programs are really useful and powerful, and they’ll remain a key part of any digital artist’s or designer’s toolset, including mine. However, for all user interface, web and icon design workflows, I recently converted to Sketch1. Here is why.
While Photoshop is awesome at what it does, defining what it is might not be so easy anymore. I remember watching a storyboarding tutorial2 by Massive Black3’s El Coro4 (unfortunately, it doesn’t seem to be available for sale anymore). In it, he says that 17 or so years ago, Adobe had no idea that digital artists were using Photoshop to digitally paint pictures! So, it had to catch up with its own user base by adding more — you guessed it — painting features.
I feel that the same kind of thing happened a bit later with Photoshop and user interface design. It was the only robust graphics tool that people knew or had access to some years ago, so they started using it for UI design, as well as for illustration and photo editing5.
Then, as a result, Adobe started adding more and more features targeted at interface designers, even though the program was initially intended and designed for a completely different purpose.
Because using Photoshop for user interface design is, in my view, a needlessly painful experience (and I am not alone10 in this view), I first tried switching to Illustrator. Illustrator made things much better for me for a few reasons: Dealing with a lot of artboards is easier; I can select any object on any layer or any artboard with a single click, without ever having to hunt down layer names in the Layers panel; also, exporting assets in the latest Creative Cloud versions has been greatly improved; and so on.
Illustrator’s vector nature has a lot of advantages for designing icons and scalable interfaces, and using it I find it much easier and quicker to draw, modify and expand shapes. It can export my assets to SVG, and it gives me control over the SVG output code. Illustrator also feels more appropriate for this type of work because the paradigm of working with vector graphics is very similar to working with web technologies (containers, CSS styles, document structure, vector fonts). After all, web browsers are essentially vector-rendering engines.
However, I hit some silly problems every now and again. For example, previewing a design on a mobile device — there is simply no software that allows me to do this directly. Photoshop has Device Preview11, but it’s for iOS only, and there is no such tool for Illustrator. So, I had to use a third-party tool named The Preview12. And every time I wanted to device-preview a screen, I had to copy my Illustrator artboard into Photoshop, scale it up (I always work in mdpi13, and my devices are xhdpi or xxhdpi), and then run The Preview on my device — which sometimes refused to work and which doesn’t support scrolling.
Also, even with the addition of CC libraries14 and with Illustrator’s dynamic symbols feature, some tasks remained kind of annoying or tedious, like using a large set of icons. In Illustrator, there is no simple and straightforward way to quickly add a premade icon and to quickly override its size or color (without creating a bunch of duplicates).
Specifying my files15 was a problem for me, too. It is, of course, my job to provide developers with a well-specified design. This helps to prevent questions such as “What is the distance between this element and that element?” and “What font size is this text object?”
Photoshop has a whole bunch of specification tools, but I’ve never used them. There aren’t that many for Illustrator, and all have their drawbacks. Arguably, the most feature-full is Specctr16, but I had technical difficulties running it (also, it has no dp as a measuring unit). So, I ended up using a slightly modified version of the Illustrator Specify script17 to measure the sizes of and distances between objects for design specifications. However, as great as that script is, specifying my designs with it was still rather tedious and time-consuming.
While these problems are arguably annoying and slow down my work, they aren’t things that I couldn’t get used to and overcome. However, the two Android projects I’m currently working on are a file manager and an email client — which means that a lot of my high-fidelity mockups are full of lists, avatars, file icons, file sizes and other types of data.
To further complicate the situation, both projects come with a light and a dark UI theme. Also, a lot of my designs need to be tailored to both phones and tablets in both landscape and portrait modes. This means a lot of copying and pasting, symbol-editing, typing, selecting and deselecting, and so on.
Admittedly, Illustrator’s symbols and libraries features can be handy and save me some manual labor. But they can be quite limited — for example, one can’t have multiple iterations of the same symbol or of the same library graphic. For me, this meant that I couldn’t use the same symbol (say, an icon or a button) in both my dark and light theme mockups because I couldn’t change its color without modifying the symbol graphic itself. The same goes for the library feature.
Problems With Data-Driven Design in Illustrator Link
Now, the most efficient way for me to avoid having to copy and paste is to have all text entries — see “Subject,” “From,” “Content Preview” and “Time” in the example below — as variables and to dynamically load predefined strings of data for each entry.
18 These are the four strings of data for my email app’s Inbox mockup.
This is, in fact, possible in Illustrator19. I could create my spreadsheet with data, import it in Illustrator with the help of a plugin20 and be ready… with one huge caveat, however: There can be only one variable entry per iteration of the file!
This makes the feature perfect for something like a business-card project, where you’d want to keep the design the same and only change some text (such as the name and phone number). When you export the final PDF files, you’d simply iterate a hundred times through a spreadsheet of data to get a hundred different names and phone numbers. This would produce a hundred nice, corporate business cards for a hundred happy employees.
However, this behavior renders Illustrator’s data features pretty much useless to me, because I need to populate a lot of of text entries simultaneously, each with a different string, on multiple artboards, and so on.
If you must stay in the Adobe Creative Cloud realm, or if switching to a Mac is not an option, there still might be a solution. Adobe’s new tool, Adobe XD21, handles working with dynamic data22 fairly well. However, because it’s a new tool, it can feel limited in other ways. Also, Adobe XD’s Windows version currently lags behind the Mac one.
Because of all of the little (and big) issues mentioned, I started to search for a better solution. So, I thought about the Sketch app.
I hadn’t had a Mac for the previous three years, and when I did have one I’d never used Sketch on it. However, I missed the operating system, and it always bothered me that Mac users always get the best software gadgets! I did plenty of research on Sketch and on other tools such as Origami, and I got tired of hacking my way through Framer6223 on Windows.
So, a couple of months ago, I decided to switch back to a Mac — both at home and at work.
Sketch cures a lot of my pains — and not just by itself, but also with the help of its vast plugin ecosystem. Compared to Illustrator and Photoshop, Sketch is focused on the needs of the UI and icon designer. And, unlike Photoshop, Sketch was made for UI design24 right from the start; UI wasn’t an afterthought.
Of course, it’s not perfect. I miss things, such as Illustrator’s Knife tool when I draw illustrations; it lacks some handy filters; there is no basic Photoshop Layer Styles alternative; and so on. But that’s kind of beside the point. Sketch doesn’t seem to be aimed at artists doing complex vector illustrations, and it only covers the very basics of raster editing.
At first, if you’ve been using Photoshop for all of your design tasks so far, you’ll be in awe when you see how you’re able to directly select any piece of graphic on any layer with just a couple of clicks or less. And if you (like me) have been also using Illustrator, then you’ll be impressed by Sketch’s more powerful Symbols feature, better artboard organization and more intuitive interface.
Measuring distances and objects is so much fun in Sketch! I just hold the Alt key, mouse over the objects on the canvas, and I’m able to see all margins, paddings and sizes. I can adjust them on the fly and just have fun, while keeping my designs really neat.
25 All margins, paddings and sizes seen at a glance
The little things are also important! One example is the ability to do simple math in an object property’s input field in the Inspector panel26. I can scale a rectangle object by typing something like 12 / 3 or 2 + 2. Very helpful sometimes.
I’ve already mentioned Sketch’s plugins. And InVision’s Craft plugins27 are probably among the most powerful and useful.
Craft actually solved my aforementioned problem of loading data into my designs. For the email app I’m working on, I can now create a list of (actual or fictitious) email addresses, subjects, content strings and more in a spreadsheet. I only need to make sure that every spreadsheet column’s first entry serves as the title (for example, “Content”).
Unfortunately, the Craft plugin doesn’t support importing data in XLS, CSV or ODS format. It only supports JSON (a file format specifically for exchanging data). I use Google Spreadsheets, and because it doesn’t support direct exporting to JSON, I have to first save to a CSV file, then use an online CSV-to-JSON converter to get the format I need.
The entire conversion process might sound complicated, but it only takes a couple of minutes.
Now, I only need to tell Craft which Sketch text field corresponds to which JSON string. Then, I use the duplicate functionality to make as many duplicate entries as I want:
As for those avatars with letters inside, I don’t need to bother with a spreadsheet. Here’s what I did instead: I used a nested symbol36 to tint each avatar with a different color. And Sketch’s overrides feature does the job of dynamically replacing each letter!
37 Sketch’s native overrides feature is what makes its dynamic symbols so powerful38.
This approach works well for expanding phone designs into tablet designs because I only need to adjust Sketch’s text fields.
39 Your data is there! Simply adjust the width of the text fields according to your needs.
As mentioned, the projects I’m currently involved in come with a light and a dark theme. Having to provide designs for both themes and for each screen can be tedious and time-consuming. However, using Sketch’s Style Inventory40 plugin, I can select all items in all of my artboards by name or by current color or by another property, and then batch-adjust them as I like.
41 Batch-selecting design elements is super-easy with the Sketch Style Inventory plugin. (View large version42)43 The dark theme mockup with the exact same data.
Thus, I can recreate my designs for different themes in just a minute!
Adding a Material icon (or an icon from another font icon set) is a no-brainer with the Sketch Icon Font plugin44. It takes only seconds! I don’t have to leave Sketch, and the icon can be scaled, colored and edited in any way I like.
45 The entire Material icon set is at my disposal with just a couple of clicks. (View large version46)
Sketch includes previewing functionality called Mirror47. However, it works only with iOS devices and web browsers (for web design mockups). Skala Preview48, on the other hand, provides excellent device-previewing functionality and (what is most important to me at the moment) support for Android devices.
Skala by itself does not support Sketch directly, but there is the Sketch Preview49 plugin to help with that. It’s reliable and has never crashed for me, but every now and again I have connection issues. What I love about it is the simple scrolling functionality that The Preview and Photoshop combination lacks.
Sketch has functionality similar to Photoshop’s and Illustrator’s swatches. It’s called global and document colors, but it has some minor limitations. For example, the palette colors can’t have names; they can’t be organized in subfolders; and no commonly used palettes come preloaded in Sketch. Also, I think the margins around the color squares are a bit too wide, so a large palette might not fit comfortably in there.
50 Sketch’s native palette isn’t that comfy for some use cases.
The Swatches51 plugin improves the situation. It comes preloaded with plenty of standard palettes, such as the Material palette, Pantone and others. I can mouse over a color to see its name, and I can apply it as a fill or stroke color or copy its HEX value with just a click.
52 The Swatches plugin with the Material design palette
Last, but not least, I need plugins to create design specifications. One of them is Zeplin.
Zeplin53 is a cloud-based toolset for creating automated design specifications and for designer-developer collaboration. Its main feature is that it reads and parses all of your Sketch artboards in a file, uploads them to a web server, and provides a web interface for other collaborators to view them. Once your artboards have been crunched and uploaded, anyone who checks them out on a computer can click and hover about and see all margins, paddings, fonts, HEX codes and other properties of all objects in the design.
This approach to design specifications is quite helpful because it saves the designer from the time-consuming task of doing specifications manually. It could also reduce confusion and misunderstanding with the development team.
Other features of Zeplin that I find quite helpful are: organization for multiple projects, screen versioning (kind of like Git55 but designed for screen mockups), project style guidelines, and optimized asset exporting.
A potential downside of Zeplin is that some companies may have concerns over its cloud-based nature.
Zeplin is good for managing and working on entire projects that contain many screens. However, if you’re looking for a simpler (and free) solution, you may want to choose something else: Sketch Measure56.
Using this plugin, you can specify your own design document or let the plugin do the dirty work. Like Zeplin, it will export a web page with your design on it, together with an interactive specification overlay! Unlike Zeplin, everything is done locally, but you won’t have the extra project-management tools at your disposal.
With so many plugins, one needs a reliable way to find, install, manage and update them. Sketch’s built-in manager only lists the plugins that are already available, allowing the user to enable and disable each one. This can be useful when you want to debug a plugin that’s acting up, but having to manually search for and install new plugins can be a hassle.
The best solution to this problem is probably the Sketch Toolbox58 app, a simple yet very useful tool. It allows you to install (and uninstall) almost any Sketch plugin out there with a single click. Highly recommended!
Sometimes, static mockups don’t quite cut it if you need to communicate a more complex idea to developers, particularly for motion design and microinteractions. Some very good tools for interactive prototyping61 are out there, and each takes a different approach to the task.
Framer takes the programming approach: The designer is required to program their prototype in CoffeeScript63. The prototype is immediately previewable in Framer Studio and can also be viewed on actual devices or in the browser.
I personally like the coding approach to creating prototypes, despite the steep learning curve. It will pay off eventually, because programming gives me a lot of control, and it also forces me to think a bit like a developer, which can be an advantage.
64 Meet Framer. On the left side is the code editor; on the right is my preview; and in the middle is the layer structure of the prototype (or of the imported Sketch file). (View large version65)
Including Framer in my Sketch workflow was fairly easy because it integrates by default. I only need to import the Sketch design file I’m working on, and I can immediately start to animate or otherwise manipulate any group of layers. The Framer-Sketch integration supports multiple artboards, but you can only import a single page at a time.
I’m still figuring out the most efficient way to work with Sketch and Framer. Currently, I think it’s best to have large per-project Sketch files, with separate pages for each screen of the app (or website) I’m working on. This enables me to easily reuse and organize all project assets, such as icons and buttons, into dynamic symbols. However, this approach produces larger files and slows down the Framer importing process. So, usually I’ll copy and paste the elements or screens that I want to prototype into a new file and then import them from there.
How Photoshop And Illustrator Fit My Workflow Now Link
I’ve moved to Sketch, but Photoshop and Illustrator are still part of my toolbox.
Because Sketch is a tool for UI design, it’s not the best-suited for creating very complex illustrations, and it doesn’t easily replace a solid vector application for illustration (nor does it attempt to).
66 While Sketch can be used to create complex illustrations, that is not its main purpose. (Source: illustration made in Sketch by Mirko Santangelo67) (View large version68)
I currently use Illustrator when I want to use my Wacom tablet for freehand drawing work, or if I’m doing something that requires complex artistic brushes and filters, or if I need to trace a raster image and convert it vectors. And, though nowadays I rarely do vector work for print (posters, leaflets, etc.), if I have to, I would again choose to use Illustrator.
Now I exclusively use Sketch for all interface and icon design tasks, but I continue to use Photoshop when I need to edit or refine product imagery, when I’m in the mood for some digital painting or drawing, or when I want to enhance a few photos. It has also happened a few times that I’ve opened Photoshop to quickly stitch together a few UI design mockups, so that I could send design previews for approval.
Sketch has solved many of my problems and has made my day-to-day life as a user interface designer a lot better. Mundane little things such as measuring distances and sizes are now much easier and quicker for me. I can now automate parts of my workflow and use real data in my Sketch designs. I can also organize my files more optimally (with the help of pages, artboards, states and symbols); I can reuse assets (through the dynamic symbols feature); and more.
Sketch also has a vibrant community. All of the open-source (and mostly free) plugins that completely transform the app and add some excellent functionality make Sketch a very versatile tool for UI design!
I really like Sketch’s focused approach, and hopefully I’ll continue to discover tiny features, tricks and plugins that make me go, “Whoa, that is so cool!”
If you’re a UI designer and are still using mostly Photoshop or Illustrator, I highly recommend you try Sketch. You might never want to look back!
Visual app platform for rapidly building real iOS & Android apps on the web. Drag and drop design, add REST APIs, and download clean Swift or Java source code.
Sometimes you have to step back and ask why a tradition exists. In mobile-first design1, serving an image in three sizes — one for smartphones, one for tablets and one for desktops — using media queries and responsive images has become a tradition. But is it the best solution?
It’s most likely better than doing nothing, but how well does it actually work? And is there room for improvement? In this article, we’ll look closely at how well the one-size-per-form-factor approach really works and how we can use smart content delivery networks to improve image performance.
Performance is a complex aspect of making a website. Here, we’ll focus on images because it’s low-hanging fruit on the tree of performance. The effort you spend on optimizing image delivery will most likely have visible and measurable results — especially if you have a website with many images, such as an e-commerce website or an online news website.
Over time, websites have become more image-heavy. Research by Radware6 indicates that the average e-commerce web page is over 1.3 MB; 64%7 of that payload comes from images. More than half of your users (and potential customers) will abandon your website if it takes longer than three seconds8 to load. Literally millions of dollars are at stake in this question of how images affect web performance.
The three image sizes referred to above are usually implemented with responsive images (and a polyfill) or with JavaScript that hijacks image-loading.
The latter was the way to go9 until responsive images came around. The idea was that the browser shouldn’t load the images by reading the HTML, but instead should load and execute JavaScript to figure out which images to download. Usually, it would query the viewport’s width, compare that width to the static breakpoints and then select the best fit out of a few predefined image sizes. As we now know, this approach has two major problems. First, it breaks browser preloading10; the browser can’t start loading images referenced in the markup right away, rather having to wait for the JavaScript to execute. Secondly, there is a significant risk of an oversized version of an image being downloaded because you have only a few predefined image sizes to pick from.
Replacing this JavaScript-based image-loading with the new responsive images11 specification addresses the preloading issue. However, the risk of an oversized version of an image being downloaded is just as significant if you have breakpoints for only three image sizes.
Picking the right, and right number of, breakpoints is not an easy task. Even if tools12 exist to help you in the process, breakpoints tend to be a moving target. The ideal breakpoints today could change tomorrow due to new screen sizes and form factors.
How serious is the problem of breakpoints for images? Asked differently, how many versions do you need to save more? Over time, the number of image requests will increase the variety of devices and screen sizes visiting your website.
An experiment done by ScientiaMobile13 found that it takes on average only eight requests for a given image to surpass three breakpoints. The experiment collected data over four months and compared the size of the image actually served to the optimal size for the particular device and screen size. Due to the wide diversity of devices of different forms and shapes accessing the web over time, statistically, the ninth image request will require a size that does not exist and most likely will get the performance penalty of downloading a larger and heavier image than necessary. The more requests you get for a given image’s URL, the more fragmented will be the devices making the requests: At 180 requests, you will surpass 11 breakpoints. At 1,000 image requests, you will surpass 20 breakpoints.
14How far will three sizes of an image get you? The actual number of requests for an image will vary according to traffic to the website. The curve shown above represents an equation that statistically best fits the image request and breakpoint data collected. (Image: ScientiaMobile2015) (View large version16)
Note that this experiment only covers image sizes. Especially with responsive images, you should also consider alternative image formats (such as GIF, JPG, PNG and WebP) for more efficient file compression. For example, you could send WebP to devices supporting it and PNG to others. This means you would need to render a few formats for each image size. You can see how this quickly multiplies image versions and requires additional logic to serve the appropriate version.
The experiment further explains17 that the strategy of using media queries and breakpoints to serve three different images to different device sizes reduces the payload served down to 63% of the original scenario in which only one size is served to all devices. But we can go further: Introducing content negotiation and server-side image optimization and resizing would reduce the payload to 16%!
Dynamically optimizing images for any device and screen size will, according to the experiment above, reduce the payload by about 75% compared to using the three static breakpoints.
As we’ve seen, displaying an image on a web page might not be as easy as it sounds if your performance budget is tight. There are many factors to account for. Picking the best breakpoints, image formats, image sizes, compression rate and implementation is challenging enough, but the solution should also be future-friendly. As new devices and form factors are launched into an already diverse landscape, supporting legacy devices becomes just as important as supporting the latest fashion. Automating this process seems to be worthy of serious consideration.
We are starting to see several turnkey solutions that optimize images in this manner. These solutions are smart content delivery networks (CDNs), and they have intelligence built in at the edge of the network. These CDNs offer device-aware image optimization by combining three elements: device detection, real-time image optimization and a classic CDN cache geographically close to the end user. Unlike regular CDNs that do not have an interest in minimizing your payload, smart CDNs seek to improve web performance by tailoring your payload to the capabilities of the device and even network conditions.
What options do you have if you want to implement a dynamic image-optimization CDN on your website? Be aware that there are a lot of image-manipulation services for performing static operations such as resizing, cropping and filtering. This is not what we’re looking for. Nor are we looking for a service that uses JavaScript to determine the best size and format of an image.
We’re looking for a CDN that implements server-side logic, or “edge logic,” using client hints22 or device detection to determine the best image size and format. This is known as content negotiation23. This makes the list significantly shorter. Let’s compare the most prominent contenders:
All of the above use client hints to determine the best size. Client hints is a fairly new initiative, currently implemented only in Blink-based browsers, and it includes some additional information about preferred image sizes in the HTTP request. The server can use this information to generate a properly sized image. Even though support and adoption of client hints are growing, only about 3% of image requests27 come with client hints. This number is expected to grow. In the near future, however, it would be a good idea to pick a CDN with device detection built in, so that all images are optimized. If you want to play around with the concept a bit, look for a pull-based CDN (which requires less configuration) with a trial option or a free tier.
All of the CDNs mentioned above require registration. Cloudinary have a free tier. ImageEngine a and Imgix has a trial concept, which is convenient if you want to try before you buy. Let’s have a closer look at ImageEngine. ImageEngine is pull-based, which means you don’t have to upload your images anywhere before you start. With ImageEngine, you simply keep images on your server, and ImageEngine will pull images from there on demand.
Once registered for ImageEngine28, you will get a dedicated hostname, which is your CDN’s URL. The only thing left to do is prefix all of your image sources with that hostname.
In this example, ImageEngine makes use of client hints and device detection to determine the optimal pixel size, compression ratio and image format for each device requesting the image. If you want to be more specific, all services listed above support explicit “commands” to override any automatically detected parameters. For ImageEngine, requesting a 360-pixel-wide image in WebP format would look like this:
ImageEngine even has a WordPress plugin29, which handles all of this automatically for you.
The process for imgix and Cloudinary is similar, aside from some additional setup.
All of the services can, of course, be combined with responsive images. With any CDN that supports client hints or device detection, the sometimes verbose markup for responsive images becomes much more maintainable. Responsive images with client hints is covered in greater detail in the article “Leaner Responsive Images With Client Hints30” here on Smashing Magazine.
Moving from one image for all kinds of devices to the common one-size-per-form-factor approach is definitely a step in the right direction. The downside is that, from a performance perspective, the approach is too general. There is more juice to be squeezed. However, from a development and maintenance perspective, it might make sense because three image sizes, or breakpoints, are manageable. The logistics are still fairly easy to maintain. But we must not forget why we do this. We’re not doing this for ourselves as developers, but for our end users. Hence, automating this process if we can makes sense. This is not a job for humans. Offloading this task is a win-win: easier maintenance and less data transfer.
Smart CDNs give you core CDN functionality, as well as dynamic and automatic image optimization for any size and form factor by using client hints and device detection at the edge of the network. Experiments suggest that payload savings can run as high as 84% compared to serving one static image, and run around 75% compared to the common one-size-per-form-factor approach.
Luckily, a few smart CDNs are out there already. It’s fairly easy to get started and measure the difference. Once you’ve created an account, you can put up a simple page and run it through WebPagetest31 to see the difference.
When you develop a game, you need to sprinkle conditionals everywhere. If Pac-Man eats a power pill, then ghosts should run away. If the player has low health, then enemies attack more aggressively. If the space invader hits the left edge, then it should start moving right. Usually, these bits of code are strewn around, embedded in larger functions, and the overall logic of the game is difficult to see or reuse to build up new levels.
Apple’s GameplayKit has several algorithms and data structures that make it easier to follow game development best practices. One of them, GKRuleSystem, lets you build up complex conditional logic from smaller pieces. By structuring your code around it, you’ll create rules that are easier to change or reuse for new levels. In this article, we’re going to take typical game logic code and learn how to represent it as a rule system.
Puzzle Games Are Made Of Lots Of Similar Levels Link
I love puzzle games. The good ones start by teaching you how the game world works. Then, along the way, you discover new capabilities and apply them to harder challenges. The developer needs to balance each level to make sure that you never get bored or want to give up. Two of my favorites are Monument Valley5 and Hundreds6. A newer iOS game, Mini Metro7, is a subway simulation game and looks perfect for me. The developers say players need to “constantly redesign their line layout to meet the needs of a rapidly-growing city,” which is a good description of the progressive challenges I’m looking for.
These games have lots of levels, each one a little harder or with a new twist. You’d think that it would be easy to build up the code for successive levels from previous ones, but as you’ll see, it can be harder than it looks.
I recently started to make a game like this called Puz-o-Mat. The Puz-o-Mat is a box with five colored buttons on it. The goal of each level is to get all of the buttons to light up by discovering the pattern that satisfies the rules of the level. Puz-o-Mat will give you feedback if you are on the right track and buzz and flash its lights to reset the level if you make a mistake.
To see why we might need a rule system, it’s useful to try to implement the game levels as a set of evaluation functions first. You can follow the code below in a Playground on GitHub12.
The goal of the first level of Puz-o-Mat is to press each button once. When you press a button, it lights up to let you know that you are on the right track. If you press one that is lit up already, the game will reset and the lights will turn off. When you tap each button once, you win the level.
Since we have a finite and defined set of game outcomes, we can just list them in an enum called GameOutcome:
enum GameOutcome: String { case win case reset }
And then, define an evaluation function that returns a GameOutcome given the buttons. If there is no outcome yet, it returns nil.
func evaluateLevel001(buttons: [Button]) -> GameOutcome? { var foundUntappedButton = false for button in buttons { if button.tapCount >= 2 { return .reset } else if button.tapCount == 0 { foundUntappedButton = true } } if !foundUntappedButton { return .win } return nil }
It’s not too hard to understand, but it’s concerning that we need a loop and three conditionals to describe the easiest level.
The second level of the game is a little harder to complete. Instead of pressing any button you want, you have to press them in order. Here’s how to do that:
func evaluateLevel002(buttons: [Button]) -> GameOutcome? { var foundUntappedButton = false for button in buttons { if button.tapCount >= 2 { return .reset } else if button.tapCount == 0 { foundUntappedButton = true } else if button.tapCount == 1 && foundUntappedButton { return .reset } } if !foundUntappedButton { return .win } return nil }
This code has just one extra else if statement. It would be nice to share some code with the previous evaluator.
As we go on from level to level, you’ll find that a line here or there may be duplicated, but it’s hard to come up with a function that handles all levels. You could do it by taking another parameter, but this game is going to have 100’s of levels; we can’t keep adding parameters and extra conditionals for each of the variations.
Moving on, the next level needs you to tap the buttons in reverse order. The function looks exactly like the last one, except the loop looks like this:
for button in buttons.reversed()
The reverse level being won in Puz-o-Mat
Again, a lot of code is the same, but it’s awkward to reuse.
There are some patterns, though.
Each evaluator starts with some game state.
Most of the work is checking conditionals against the game state to see what we should do next. There are many different conditions across the whole game, and each level seems to mix and match them.
The main point of the evaluator is to decide if we need to reset or win.
Even though there is a pattern to the level game logic, all of the parts are mixed together and aren’t easily separated for reuse.
Using this insight we could try to restructure the code. A promising place to start is by pulling out the conditionals into separate functions. As we’ll see later, this is the design of GameplayKit’s rule system, but we can get part of the way there by playing with this code first. Then, we’ll convert our result to GameplayKit.
First, let’s use a type alias to define what a rule is:
typealias Rule = (_: [Button]) -> GameOutcome?
A rule is a function that takes the array of buttons and responds with an outcome if it knows what to do or, nil if not.
Many levels may limit the number of taps on a button, so we’ll make a function to return a rule that checks buttons against a tap limit:
Then, we need loop through all of the rules and run them so that they can check their conditional against the button state. If any of the rule functions return a GameOutcome, we’ll return it from the evaluator. If none are true, we’ll return nil.
Following this train of thought, you could expand the rules to take in more parameters or check more states. Each level is expressed as a list of rules that produce some outcome. Rules are separately encapsulated and easily reused.
This is such a common way of structuring algorithms in game logic, that Apple provides a set of classes in GameplayKit that we can use to build games in this style. They are collectively known as the Rule System classes and are primarily implemented in GKRuleSystem14 and GKRule15.
Using these Rule System classes hides all of the complexity of the evaluator, but more importantly, delivers much more power than our simple one.
In the GameplayKit rule system, you need to define three things:
The game state
A dictionary that will represent your game state. It can have any keys and structure that you wish.
A set of fact objects
The possible results of evaluating the rules. Facts can be any object, but in Swift it makes sense for them to be an enum. We could use the GameOutcome enum we used in our examples above.
A list of rules
The GKRule objects to evaluate. They each provide a conditional to check against the state and an action to perform if the conditional is true.
Note: GameplayKit classes were designed for Objective-C, which is why I had to derive GameOutcome from String and use the enums as rawValues.
This is a very simple use of GKRuleSystem, and it’s fine if you don’t need to do this in the context of an action game (and need to keep up with a high frame rate). In that case, you can use the fact that the rule system state property is a mutable dictionary that you can alter directly rather than recreate. You could also reuse rule system objects with rules set up.
This is similar to our evaluator from the last section, but this one can operate over a more complex state object. Also, GKRuleSystem.evaluate() is more sophisticated than a loop over the rules. You can provide rule precedence, fact retraction (rules that reverse fact assertions), and find out which exact rules were invoked, among other features.
The last step is converting our Rule functions to GKRule. You could derive GKRule subclasses for each rule, but for most applications, the GKRule constructors will be good enough.
During the GKRuleSystem evaluation, this object will have its first block called to see if the rule should be applied. The block could refer to external sources, but the most common thing is to look at the passed in rule system’s state property and check it for a condition. If the first block returns true, then the second one is called. It could also do anything, but it’s expected that it would assert or retract facts.
This is a good constructor to use if you have complex state and conditionals, or if you are using information outside of the rule system to implement rules. Since the expectation and normal case is to use the state and make facts, there is a more direct constructor to use.
The only thing you need for a new level is a new list of GKRules. We have gone from an imperative level description where we need to give every step to a declarative one where we provide queries and outcomes.
Another benefit of this approach is that the predicates are serializable, so they can be stored in external files or sent over the network. With just a few more lines of code, we could move the predicate and enum to a .plist file and load them instead of hard-coding them. Then, a game designer on your team could make or tweak levels without editing code. The main coding work would be to add more state and outcomes.
In this example, facts are mutually exclusive; you can either win or reset, but not both. But, this is not a restriction of rule systems. If it makes sense for multiple facts to be asserted, then you may do so, and the code that checks the facts will need to reconcile them. One example is that we could treat each button’s light as a fact (on or off) and there would be a fact asserted for each button as the game was played. In that case, there could certainly be many facts asserted at the same time.
Finally, you may have noticed that facts are asserted with a grade. This grade is a number from 0 to 1 that you can think of as a probability that this fact is correct. We used 1 to indicate certainty, but you could use lower numbers to indicate that the fact only has a probability of being true. These probabilities can be combined and, using random numbers, we could pick among the asserted facts and have emergent, rather than deterministic, game behavior. This is known as Fuzzy Logic, and I’ll cover that in a future article.
This article concentrated on just the game logic aspect of developing a game for iOS, but if you want to see how to construct the visual aspect of a game using SpriteKit, check out this series right here on Smashing: Part I23, Part II24, Part III25.
This WWDC video26 (Safari required) covers GameplayKit. Go to 43:10 in the video to hear about Rule Systems.
As a designer, you will be facing more demands and opportunities to work with digital systems that embody machine learning. To have your say about how best to use it, you need a good understanding about its applications and related design patterns.
This article illustrates the power of machine learning through the applications of detection, prediction and generation. It gives six reasons why machine learning makes products and services better and introduces four design patterns relevant to such applications. To help you get started, I have included two non-technical questions that will help with assessing whether your task is ready to be learned by a machine.
Big data and big promises. We are expecting a great many things to happen once the big data deluge has been funnelled into a nurturing stream of bits. Data can be used in many ways. One is to build smart products, and another is to make better design and business decisions. The latter also, ultimately, trickle into products.
Machine learning is a very promising approach radically shaping future product and service development. Machine learning is a branch of artificial intelligence. It employs many methods: Deep learning and neural networks are two well-known instances.
Machine learning means that, instead of programmers providing computers with very detailed instructions on how to perform a task, computers learn the task by themselves. A recent, remarkable milestone was when Google’s AlphaGo software learned to master the game of Go at the level of a world champion!
Pretty much anything that a normal person can do in <1 sec, we can now automate with AI.
This story will lead you into problem discovery through examples of what sorts of problems machines today readily chew on. A basic understanding of machine learning, commonly known as ML, will help. If you are unfamiliar with ML, I suggest you read my friendly introduction6 to the topic or Nvidia’s clear explanation7 of the differences between ML, AI and deep learning.
Why Do I Need To Understand The Applications Of Machine Learning? Link
As a designer, you will be facing more demands and opportunities to work with digital systems that embody machine learning. As the hype around machine intelligence intensifies, this will lead to technology-driven pressure to extensively utilize machine learning. This may happen with little understanding of its actual benefits and its impact on product desirability and customer experience.
Stay on Top of the Machine-Intelligence Game! Link
As a designer, to have your say about any plans for machine intelligence and how it is best implemented on the human interface layer, you need to know what it can do and how digital services can utilize it. This post provides an overview of applications, with concrete examples, as well new design guidelines that you can put to work. This will help with making actual design decisions and identifying the right design patterns, including situations when no directly applicable solution exists and you must transfer ideas across domains.
To exploit the capability of machine learning, you must grasp the nature of a task as a computer might see it. The concept I’ll use to describe it is the core problem of learning. This refers to defining what exactly we would like the computer to learn in order for it to complete the task we have assigned to it. These goals are not always evident at the practical, holistic level of a finished product (say, Tesla’s autopilot function). This story will help you to see what goes on under the surface.
Say you want the computer to drive your car.
This is a very high abstraction level learning goal. We need to go further down and break it into smaller chunks. We need to ask questions such as, can the computer accelerate and decelerate, and can the machine recognize red and green lights? To identify the core problem is already a move towards understanding whether the overarching task is a good fit for a machine to learn. Once the core problem of learning is defined well, then it is possible to say whether the ML computers of today can solve it with adequate accuracy and in a decent amount of time. This is what matters most for actually making the application work.
8Small autonomous buses (in the background) started operating on Aalto University Espoo campus in autumn 2016. The street sign warns about their presence, reminiscent of the first automobiles in the 19th century. (Picture of Project Sohjoa9 demonstration in Espoo. Copyright Metropolia UAS, used with permission.) (View large version10)
Suppose you manage to teach some skill to a machine. How would your product or service benefit from it? Here are six possible benefits:
augment,
automate,
enable,
reduce costs,
improve safety,
create.
In rare cases, machine learning might enable a computer to perform tasks that humans simply can’t perform because of speed requirements or the scale of data. But most of the time, ML helps to automate repetitive, time-consuming tasks that defy the limits of human labor cost or attentional span. For instance, sorting through recycling waste 24/7 is more reliably and affordably done by a computer.
In some areas, machine learning may offer a new type of expert system that augments and assists humans. This could be the case in design, where a computer might make a proposal for a new layout or color palette aligned with the designer’s efforts. Google Slides already offers this type of functionality through the suggested layouts feature11. Augmenting human drivers would improve traffic safety if a vehicle could, for example, start braking before the human operator could possibly react, saving the car from a rear-end collision.
Google Slides provides design assistance via the Explore function. Right pane demonstrates the variations it has generated from elements initially composed by the user.
In 2016, the most celebrated milestone of machine learning was AlphaGo’s victory14 over the world champion of Go, Lee Sedol15. Considering that Go is an extremely complicated game to master, this was a remarkable achievement. Beyond exotic games such as Go, Google Image Search is maybe the best-known application of machine learning. Search feels so natural and mundane when it effectively hides away all of the complexity is embeds. With over 30 billion search queries every day, Google Image Search constantly gets more opportunities to learn.
There are already more individual machine learning applications than are reasonable to list here. But a major simplification16 is not sufficient either, I feel. One way to appreciate the variety is to look at successful ML applications from Eric Siegel’s book Predictive Analytics from 2013. The listed applications fall under the following domains:
marketing, advertising and the web;
financial risk and insurance;
healthcare;
crime fighting and fraud detection;
fault detection for safety and efficiency;
government, politics, nonprofit and education;
human-language understanding, thought and psychology;
staff and employees, human resources.
His cross-industry collection of examples is a powerful illustration of the omnipresence of predictive applications, even though not all of his 147 examples utilize machine learning as we know it. However, for a designer, knowing whether your problem domain is among the nine listed will give an idea of whether machine learning has already proven to be useful or whether you are facing a great unknown.
Machines Learn Detection, Prediction And Creativity Link
As I see it, the power of learning algorithms comes down to two major applications: detection and prediction. Detection is about interpreting the present, and prediction is about the way of the future. Interestingly, machines can also do generative or “creative” tasks. However these are still a marginal application.
17Three main categories of ML applications and their common use cases (View large version18)
When you combine detection and prediction, you can achieve impressive overall results. For instance, combine the detection of traffic signs, vehicles and pedestrians with the prediction of vehicular and pedestrian movements and of the times to vehicle line crossings, and you have the makings of an autonomous vehicle!
This is my preferred way of thinking about machine learning applications. In practice, detection and prediction are sometimes much alike because they don’t yet cut into the heart and bones of machine learning (recall the basics of machine learning19), but I believe they offer an appropriate level of abstraction to talk about machine learning applications. Let’s clarify these functions through examples.
Text and speech are the most natural interaction and communication methods. Thus, it has not been feasible in the realm of computers. Previous generations of voice dialling and interactive voice response systems were not very impressive. Only in this decade have we seen a new generation of applications that take spoken commands and even have a dialogue with us! This can go so smoothly that we can’t tell computers and humans apart in text-based chats, indicating that computers have passed the Turing test20.
Dealing with speech, new systems such as personal assistant Siri or Amazon’s Echo device are capable of interpreting a wide range of communications and responding intelligently. The technical term for this capability is natural language processing (NLP). This indicates that, based on successful text and speech detection (i.e. recognition), computers can also interpret the meaning of words, spoken and written, and take action.
Text interpretation enables equally powerful applications. The detection of emotion, or sentiment, from text means that large masses of text can be automatically analyzed to reveal what people in social media think about brands, products or presidential candidates. For instance, Google Translate just recently witnessed significant quality improvements21 by switching to an ML approach to translations.
22Amazon Echo Dot is surfacing as one of the best-selling speech-recognition-driven appliances of early 2017. (View large version23)
Computer vision gives metaphorical eyes to a machine. The most radical example of this is a computer reconstruction of human perception from brain scans! However, that is hardly as useful an application as one that automates the tagging of photos or videos to help you explore Google Photos or Facebook. The latter service recognizes faces to an even scary level of accuracy.
Image interpretation finds many powerful business applications in industrial quality control, recording vehicle registration plates, analyzing roadside photos for correct traffic signs, and monitoring empty parking spaces. The recent applications of computer vision to skin cancer diagnosis27 have actually proven more proficient than human doctors, leading to the discovery of new diagnostic criteria!
28A search for “dog” in my personal Google Photos collection brings up several correct instances of dogs I’ve chanced upon, but also a few false positives. However, I’ve never tagged a single dog, so the noise is acceptable given the added value of finding any dogs. (Screenshot from February 2017, edited to remove personal information (albums).) (View large version29)
Speech was already mentioned, but other audio signals are also well detected by computers. Shazam and SoundHound have for years provided reliable detection of songs either from a recording fragment or a sung melody. The Fraunhofer Institute developed the Silometer30, an app to detect varieties of coughs as a precursor to medical diagnosis. I would be very surprised if we don’t see many new applications for human and non-human sounds in the near future.
Given that computers are seeing and hearing what we do, it is not surprising that they have became capable of analyzing and detecting human behavior and identity as well — for instance, with Microsoft Kinect recognizing our body motion. Machines can identify movements in a football game to automatically generate game statistics. Apple’s iPad Pro recognizes31 whether the user is using their finger or the pencil for control, to prevent unwanted gestures. A huge number of services detect what kind of items typically go together in a shopping cart; this enables Amazon to suggest that you might also be interested in similar products.
In the world of transportation, it would be a major safety improvement if we could detect when a driver is about to fall asleep behind the steering wheel, to prevent traffic accidents. Identity detection is another valuable function enabled by several signals. A Japanese research institute has developed a car seat32 that recognizes who’s sitting in it. Google’s reCAPTCHA33 is a unique system that tells apart humans from spam bots. Perhaps the most notorious example of guessing people’s health was Target’s successful detection of expectant mothers34. This was followed by a marketing campaign that awkwardly disclosed the pregnancy of Target customers, resulting in much bad publicity.
35Google’s reCAPTCHA has simplified spam-fighting in web forms with the help of micro-intelligence from machine learning.
Machine learning is also used to detect and prevent fraudulent, abusive or dangerous content and schemes. It is not always major attacks; sometimes it is just about blocking bad checks or preventing petty criminals from entering the NFL’s Superbowl arena36. The best successes are found in anti-spam; for instance, Google has been doing an excellent job for years of filtering spam from your Gmail inbox.
I will conclude with a good-willed detection example from the normal human sphere. Whales can be reliably recognized37 from underwater recordings of their sounds — once more, thanks to machine learning. This can help human-made fishing machinery to avoid contact with whales38 for their protection.
39This humpback whale would be all the merrier if all fishing boats were equipped with machine-learning technology to detect their signals. (View large version40)
Design Pattern: Suggested Features
Text and speech prediction have opened up new opportunities for interaction with smart devices. Conversational interfaces are the most prominent example of this development, but definitely not the only one. As we try to hide the interface and underlying complexity from users, we are balancing between what we hide and what we reveal. Suggested features help users to discover what the invisible UI is capable of.
Graphical user interfaces (GUIs) have made computing accessible for the better part of the human race that enjoys normal vision. GUIs provided a huge usability improvement in terms of feature discovery. Icon and menus were the first useful metaphors for direct manipulation of digital objects using a mouse and keyboard. With multitouch screens, we have gained the new power of pinching, dragging and swiping to interact. Visual clues aren’t going anywhere, but they are not going to be enough when interaction modalities expand.
How Does a User Find Out What Your Service Can Do? Link
Haptic interaction in the first consumer generation of wearables and in the foremost conversational interfaces presents a new challenge for feature discovery. Non-visual cues must be used that facilitate the interaction, particularly at the very onset of the interactive relationship. Feature suggestions — the machine exposing its features and informing the user what it is capable of — are one solution to this riddle.
In the case of a chatbot employed for car rentals, this could be, “Please ask me about available vehicles, upgrades and your past reservations.”
Specific application areas come with specific, detailed patterns. For instance, Patrick Hebron’s recent ebook41 from O’Reilly contains a great discussion of the solutions for conversational interfaces.
Several generations of TV watchers have been raised to watch weather forecasts for fun, ever since regular broadcasts began after the Second World War42. The realm of prediction today is wide and varied. Some applications may involve non-machine learning parts that help in performing predictions.
Here I will focus on the prediction of human activities, but note that the prediction of different non-human activities is currently gaining huge interest. Predictive maintenance of machines and devices is one such application, and more are actively envisioned as the Internet of Things generates more data to learn from.
Predicting different forms of human behavior falls roughly into the following core learning challenges and applications:
recommendations,
individual behavior and condition,
collective behavior prediction.
Different types of recommendations are about predicting user preferences. When Netflix recommends a movie or Spotify generates a playlist of your future favorite music, they are trying to predict whether you will like it, watch it or listen through to the end of the piece. Netflix is on the lookout for your rating of the movie afterwards, whereas Spotify or Pandora might measure whether you are returning to enjoy the same song over and over again without skipping. This way, our behaviors and preferences become connected even without our needing to express them explicitly. This is something machines can learn about and exploit.
In design, predicting which content or which interaction models appeal to users could give rise to the personalization of interfaces. This is mostly based on predicting which content a user would be most interested in. For a few years now, Amazon has been heavily personalizing the front page, predicting what stuff and links should be present in anticipation of your shopping desires and habits.
Recommendations are a special case of predicting individual behavior. The scope of predictions does not end with trivial matters, such as whether you like Game of Thrones or Lady Gaga. Financial institutions attempt to predict who will default on their loan or try to refinance it. Big human-resource departments might predict employee performance and attrition. Hospitals might predict the discharge of a patient or the prognosis of a cancer. Rather more serious humans conditions, such as divorce, premature birth and even death within a certain timeframe, have been all been predicted with some success. Of course, predicting fun things can get serious when money is involved, as when big entertainment companies try to guess which songs43 and movies will top the charts to direct their marketing and production efforts.
44 Microsoft’s ML platform demonstrator How Old Do I Look?45 guesses age (a human condition) based on any uploaded photo or search.
The important part about predictions is that they lead to individual assessment that are actionable. The more reliable the prediction and the weightier the consequences, the more profitable and useful the predictions become.
Predicting collective behavior becomes a generalization of individuals but with different implications and scope. In these cases, intervention is only successful if it affects most of the crowd. The looming result of a presidential election, cellular network use or seasonal shopping expenditure can all be subject to prediction. When predicting financial risk or a company’s key performance indicators, the gains of saving or making money are noticeable. J.P. Morgan Chase was one of the first banks to increase efficiency by predicting mortgage defaulters (those who never pay back) and refinancers (those who pay back too early). On the other hand, the recent US presidential election is a good reminder that none of this is yet perfect.
In a close resemblance to tools for antivirus and other present dangers, future trouble is also predictable. Predictive policing is about forecasting where street conflicts might happen or where squatters are taking over premises, which would help administrators to distribute resources to the right places. A similar process is going on in energy companies, as they try to estimate the capacity needed to last the night.
Design Pattern: Personalization
Once a computer gets to know you and to predict your desires and preferences, it can start to serve you in new, more effective ways. This is personalization, the automated optimization of a service. Responsive website layouts are a crude way of doing this.
The utilization of machine learning features with interfaces could lead to highly personalized user experiences. Akin to giving everyone a unique desktop and homescreen, services and apps will start to adapt to people’s preferences as well. This new degree of personalization presents opportunities as well as forces designers to flex their thoughts on how to create truly adaptive interfaces that are largely controlled by the logic of machine learning. If you succeed in this, you will reward users with a superior experience and will impart a feeling of being understood46.
47Amazon.com’s front page has been personalized for a long time. The selection offered to me looks somewhat relevant, if not attractive. (View large version48)
Currently, personalization is foremost applied in order to curate content. For instance, Amazon carefully considers which products would appeal to potential buyers on its front page. But it will not end with that. Personalization will likely lead to much bigger changes across UIs — for instance, even in the presentation of the types of interactive elements a user likes to use.
Questions To Ask In Considering The Fit Of Machine Learning Link
Say you are now convinced that your users and customers would benefit from a ML-boosted service. Next, you must consider the business and technology perspectives. The first question to ask is, If ML works at least as well as you want, what value would it add to your product? Be honest: Traditional business logic applies here, too.
If machine learning doesn’t create value, then it is probably a waste of resources. Marketing people might fancy your new implementation, but the business folks will not necessarily fund your next machine learning experiment unless you have a business case for why machine learning would improve the customer experience or revenue directly.
OK, suppose you’ve cleared the viability check box. I expect you also have at least a hypothesis of the core learning challenge. This might be, for example, Can a computer learn to predict when the user would need to take an umbrella or a waterproof jacket when they leave the house in the morning.
Next, you’ll need to figure out whether you can expect the machine to learn the job you wish to get done. We’ve come to matters of data. Data broadly refers to any information you can feed the computer: weather data, shopping data for umbrellas and waterproof gear, social media updates and so forth.
Here are two simple diagnostic questions to assess the data situation:
Do we have enough examples for the machine to learn from?
Are there patterns in the teaching material that can be recognized by a human expert?
The first question looks easy but is difficult to answer in advance. You need both good quality and a sufficient quantity of data. Some signals are noisy (that is, have poor quality). You might need more examples in these cases than in others. Some features are very prominent and easy to learn, such as in the case of detecting nighttime or daytime in photos. This can be solved with as few as 30 training images49!
Typical applications require thousands of instances of input data and possibly the desired answers. The more complicated the task, the more material will be needed. AlphaGo learned to master the game after analyzing a staggering 30 million games50 and then playing some against itself! This is likely excessive in most situations, but it gives you an idea of the scale of what computers can and may need to swallow — from 30 to 30 million. This is one of the reasons why increasing computing capacity and certain hardware really makes computers smarter by speeding up the learning of truly big data.
51The complexity of Go pushed machine learning capacity to its extremes. Playing the game at random, or by brute force, would be futile. (Image: YouTube52) (View large version53)
The second question is, Are there patterns in the teaching material that can be recognized by a human expert? If a human can do the task, then there’s a fair chance that there are regularities in the data that might be picked up by the computer as well.
If you have a positive answer to least one of the questions, you can go forward with some confidence that machine learning might indeed be of use to you. What you’d do after this discovery is mostly beyond the scope of this article.
In order to build a machine learning solution, it is best to boldly go and prototype your desired functionality. If you have enough samples, your data scientist teammate will quickly discover whether the machine shows proficiency in the learning task (particularly if you use scalable could computing). If not, you’ll need to get more data.
What follows is a series of iterations on the learning mechanisms and a process of integrating it with the product or service. The appearance of intelligent applications can be achieved by carefully putting together several pieces of machine learning and even conventional programming. This is called an ensemble approach. What at best appears as a seamless interactive experience for the user may in fact be the product of a very complex rubric of different machine learning components working together.
Human intelligence is heavily needed to tweak the learning algorithms. Such was the architecture underlying Watson54, the Jeopardy-winning artificial intelligence system created by IBM and the algorithm that claimed the Netflix Prize55. In the latter competition, several teams combined their individual tweaks during the final months of the competition to eventually exceed the criteria for success. This is also what the Google Translate team did to ascend to the next level. And that work is something I consider to fall under the domain of traditional software development.
New Design Patterns For Creating Interfaces Of Machine Learning Applications! Link
Thus far, I’ve talked about the possibilities of machine learning and given some practical advice on how to figure out its feasibility. I’ve also introduced two general design patterns that are tightly connected to machine learning. However, they are not enough.
A designer engaged in service and interface design will have several questions concerning interaction in their mind by now. How can we and will we communicate machine intelligence to users, and what kinds of new interfaces will machine learning call for?
This entails both opportunities to do things in a new way, as well as requirements for new designs. To me, this means we will need a rise in importance of several design patterns or, rather, abstract design features, which will become important as services get smarter. They include:
suggested features,
personalization,
shortcuts versus granular controls,
graceful failure.
I have already covered suggested features58 and personalization59 as a part of detection and prediction, but what are granularity and graceful failure?
Photoshop is an excellent example of a tool with a steep learning curve and a great deal of granularity in controlling what can be done. Most of the time, you work on small operations, each of which has a very specific influence. The creative combination of many small things allows for interesting patterns to emerge on a larger scale. Holistic, black-box operations such as transformative filters and automatic corrections are not really the reason why professionals use Photoshop.
What will happen when machines learn to predict what we are doing repeatedly? For instance, I frequently perform certain actions in Photoshop before uploading my photos to a blog. While I could manually automate this, creating yet another user-defined feature among thousands already in the product, Photoshop might learn to predict my intentions and offer a more prominent shortcut, or a highway, to fast-forward me to my intended destination. As Adobe currently puts effort into bringing AI into Creative Cloud, we’ll likely see something even more clever than this very soon. It is up to you to let the machine figure out the appropriate shortcuts in your application.
60A mockup of a possible implementation of “predictive history” in Photoshop CC. The application suggests a possible future state for the user based on the user’s history and preceding actions and on the current image. (View large version61)
A funny illustration of a similar train of thought comes from Cristopher Hesse’s machine-learning-based image-to-image translation62, which provides interesting content-specific filling of doodles. Similar to Photoshop’s content-aware fill, it creates most hilarious visualizations of building facades, cats, shoes and bags based on minimal user input.
63The edges2cats algorithm employs machine learning to finish your cat doodle as a photorealistic cat monster. (View large version64)
I call the final pattern graceful failure. It means saying “sorry, I can’t do what you want because…” in an understandable way.
This is by no means unique to machine learning applications. It is innately human, but something that computers have been notoriously bad at since the time that syntax errors were repeatedly echoed by Commodore computers in the 1980s. But with machine learning, it’s slightly different. Because machine learning takes a fuzzy-logic approach to computing, there are new ways that the computer could produce unexpected results — that is, things could go very bad, and that has to be designed for. Nobody seriously blames the car in question for the death that occurred in the Tesla autopilot accident in 2016.
The other part is that building applications that rely on modern machine learning is still in its infancy. Classic software development has been around for so long that we’ve learned to deal with its insufficiencies better. As Peter Norvig, famous AI researcher and Google’s research director, puts it like this65:
The problem here is the methodology for scaling this up to a whole industry is still in progress.… We don’t have the decades of experience that we have in developing and verifying regular software.
The nature of learning is such that computers learn from what is given to them. If the algorithm has to deal with something else, then the results will not be to your liking. For example, if you’ve trained a system to detect animal species from pet photos and then start using it to classify plants, there will be trouble. This is more or less why Microsoft’s Twitterbot Tay had to be silenced66 after it picked up the wrong examples from malicious users when exposed to real-world conditions.
The uncertainty in detection and prediction should be taken into consideration. How this is done depends on the application. Consider Google Search. No one is offended or truly hurt, but merely amused or frustrated, by bad search results. Of course, bad results will eventually be bad for business. However, if your bank started using a chatbot that suddenly could not figure out your checking account’s balance, you would be rightfully worried and should be offered a quick way to resolve your trouble.
To deal with failure, interfaces would do well to help both parties adjust. Users can tolerate one or two “I didn’t get that, please say that again” prompts (but no more) if that’s what it takes to advance the dialogue. For services that include machine learning, extensive testing is best. Next comes informing users about the probability and consequences of failure, and instructions on what the user might do to avoid it. The good practices are still emerging.
With machine learning, our vision of tomorrow is quickly becoming today’s reality.
Overall, there hardly seems to be an application that machine learning could not be used to detect or predict. I’ve introduced plenty of examples of deploying machine learning to varying success. In all of the examples, I’ve tried to illustrate some core learning challenge to help you understand what sorts of tasks machines respond to. Now it is time to face the question of what machine learning can do for you!
I don’t expect that this guide alone will suffice for radical innovation in the sphere of intelligent products and services. I do hope that it will open the eyes of several designers to the opportunities that machine learning solutions afford us today. It is important to understand approximately what you can realistically ask from a machine. In the near future, good questions will become ever more important, so that the answer will be, “Yes, Dave. I can do that.”
Thanks to Max Pagels, Janne Aukia, Antti Rauhala, Teemu Kinnunen and Patrick Hebron for discussing the topic.
Bots and Artificial Intelligence are probably the most hyped concepts right now. And while some people praise the existing technologies, others claim they don’t fear AI at all, citing examples where it fails horribly. Examples of Facebook or Amazon advertising (both claim to use machine learning) that don’t match our interests at all are quite common today.
But what happens if we look at autonomous cars, trains or planes that have the very same machine learning technologies in place? How about the military using AI for its actions? While we’re still experimenting with these capable technologies, we also need to consider the possible consequences, the responsibilities that we have as developers and how all of this might affect the people the technology is being served to.
This week, Firefox 53 rolled out5 to end users, shipping performance benefits, positioned CSS Masks, and the new display: flow-root value that effectively replaces our common clearfix methods6. The update also comes with a revamped media player design. Finally, this is the first Firefox version without Windows XP and Vista support, so if you rely on one of these operating systems, consider switching to the ESR version of Firefox and upgrade to a newer system as soon as possible (the OS are not supported by Microsoft anymore).
Chrome 587 comes with support for IndexedDB 2.0, fullscreen support for progressive web apps, and improvements for sandboxed iframes. Alongside Firefox 53, the new Chrome is the second browser to support display: flow-root, the new clearfix replacement8. There’s also PointerEvents.getCoalescedEvents() as a new method to give you access to all input events that took place since the last time a PointerEvent was delivered — a useful feature for drawing applications but also quite risky when we look at it from a privacy and user tracking perspective.
Mozilla finally simplified the developer experience and got rid of the Firefox Developer Edition9. If you still use it, switch to the Firefox Nightly Edition10. While there’s still a beta channel, I recommend Nightly as it’s relatively free of bugs that impact the general usage while supporting the latest features, deprecations and development tools already weeks or even months ahead of public launch. This is great as you have more time to adjust code on live sites when something breaks in the Nightly channel. I use WebKit Nightly and Chrome Canary similarly.
Peter O’Shaughnessy challenges us to estimate which web browsers have the most users11. And as you can probably assume, our existing idea of Chrome, Firefox, Safari, and IE leading the field isn’t up to date anymore. Instead, we need to acknowledge that UC Browser has an impressive market share, Opera Mini still does, too, Yandex in certain regions and Samsung Internet usage grows fast as more devices are shipped with it. And Google Analytics isn’t telling us the truth anyway — big parts of “Chrome” might actually be Samsung Internet.
Google Chrome can now be run in headless mode, replacing PhantomJS or SlimerJS. Jim Cummins explains how to set it up on Mac OS16. For Windows and Linux it should be similar using bash and a few adaptions to the local commands.
Jeremy Thomas experimented with browsers and tried to disable cookies entirely19. Read about how successful he was with it and what challenges he faced with modern web applications.
Stefan Judis started a discussion about whether it’s time to rethink bundling25 as support for ES6 modules is now landing in browsers (currently in Safari, and behind a flag in Firefox and Edge, while Chrome has it in development).
Jekyll is gaining popularity as a lightweight alternative to WordPress. It often gets pigeonholed as a tool developers use to build their personal blog. That’s just the tip of the iceberg — it’s capable of so much more!
In this article, we’ll take on the role of a web developer building a website for a fictional law firm1. WordPress is an obvious choice for a website like this, but is it the only tool we should consider? Let’s look at a completely different way of building a website, using Jekyll.
Jekyll6 is a static website generator. Instead of software and a database being installed on our server, a Jekyll website is simply a directory of files on our computer. When we run Jekyll on that directory, it generates a static website, which we upload to a hosting provider.
A Jekyll website is essentially a static website with a templating language. It has fewer components to create and maintain. On the server, we only need a web server capable of serving files.
Speed
When visitors view pages on Jekyll sites, the server returns existing files without any extra processing. This is much faster than WordPress, which generates pages dynamically at request time. Note: WordPress Caching plugins can eliminate this performance gap.
Stability
WordPress has more components working together to generate pages for visitors. If a component fails, visitors may not be able to view the website. Much less can go wrong when a web server is serving only files.
Security
WordPress does a lot to mitigate security risks such as CSRF, XSS, or SQL injection attacks however it relies on you always having the latest updates. Statics sites eliminate this problem because there’s no dynamic data storage a hacker can exploit.
Source-controlled
A Jekyll website is a directory of files, so we can store the entire website in a Git repository. Working with a repository gives us many benefits7 (although VersionPress8 is in development and enables this workflow for WordPress).
A client can sign up to WordPress.com, choose a theme and set up a basic website by themselves. Jekyll is a command-line tool, which overwhelms most non-technical users. There are third-party GUIs for Jekyll, including CloudCannon9 (disclaimer: I’m the cofounder), Forestry10, Jekyll Admin11, Netlify CMS12, Prose13 and Siteleaf14. However, these need to be set up by the developer before being handed off to the client.
Build time
In our situation, this isn’t a problem because the website will build in under a second. However, a larger website with 10,000 to 100,000 posts could take minutes to build. This is frustrating when we’re developing because we have to wait for the website to build before previewing it in the browser.
Themes
Jekyll has some themes available, but it’s nothing compared to the thousands of themes available for WordPress.
Extensibility
If we need to add custom functionality to our WordPress website, we can write our own PHP. We can create custom Ruby plugins for Jekyll, however, these run at build time rather than request time.
Support
WordPress has a huge community of experts and other resources to help. Jekyll has similar resources but on a smaller scale.
Jekyll is a great tool for largely informational websites, like this project. If the project is more of an application, we could add dynamic elements using JavaScript, but at some point we would probably need a back end like WordPress’.
A typical development environment for WordPress requires installation of Apache or NGINX, PHP and MySQL. Then, we would install WordPress and configure the web server.
For Jekyll, we need to make sure we have Ruby installed (sometimes this is harder than it sounds). Then we install the Jekyll gem:
gem install jekyll
If you’re on macOS make sure you have Xcode developer installed first.
It’s time to create our first page. Let’s start with the home page. Pages are for standalone content without an associated date. WordPress stores page content in the database.
In Jekyll, pages are HTML files. We’ll start with pure HTML and then add Jekyll features as they’re needed. Here’s index.html in its current state:
In WordPress, we can write PHP to do almost anything. Jekyll takes a different approach. Instead of providing a full programming language, it uses a templating language named Liquid15. (WordPress has templating languages, too, such as Timber16.)
The footer of index.html contains a copyright notice with a year:
We’re building a static website in Jekyll, so this date won’t change until we rebuild the website. If we wanted the date to change without having to rebuild the website, we could use JavaScript.
The bulk of the HTML in index.html is for setting up the overall layout and won’t change between pages. This repetition will lead to a lot of maintenance, so let’s reduce it.
Includes were one of the first things I learned in PHP. Using includes, we can put the header and footer content in different files, then include the same content on multiple pages.
Jekyll has exactly the same feature. Includes are stored in a folder named _includes. We use Liquid to include them in index.html:
{% include header.html %} <p>Justice Law is professional representation. Practicing for over 50 years, our team have the knowledge and skills to get you results.</p> <blockquote> <p>Justice Law are the best of the best. Being local, they care about people and have strong ties to the community.</p> <p> <img src="/images/peter.jpeg" alt="Photo of Peter Rottenburg"> Peter Rottenburg </p> </blockquote> {% include footer.html %}
Includes reduce some of the repetition, but we still have them on each page. WordPress solves this problem with template files that separate a website’s structure from its content.
The Jekyll equivalent to template files is layouts. Layouts are HTML files with a placeholder for content. They are stored in the _layouts directory. We’ll create _layouts/default.html to contain a basic HTML layout:
Then, replace the includes in index.html by specifying the layout. We specify the layout using front matter, which is a snippet of YAML17 that sits between two triple-dashed lines at the top of a file (more on this soon).
--- layout: default --- <p>Justice Law is professional representation. Practicing for over 50 years, our team have the knowledge and skills to get you results.</p> <blockquote> <p>Justice Law are the best of the best. Being local, they care about people and have strong ties to the community.</p> <p> <img src="/images/peter.jpeg" alt="Photo of Peter Rottenburg"> Peter Rottenburg </p> </blockquote>
Now we can have the same layout on all of our pages.
In WordPress, custom fields allow us to set meta data on a post. We can use them to set SEO tags or to show and hide sections of a page for a particular post.
This concept is called front matter18 in Jekyll. Earlier, we used front matter to set the layout for index.html. We can now set our own variables and access them using Liquid. This further reduces repetition on our website.
Let’s add multiple testimonials to index.html. We could copy and paste the HTML, but once again, that leads to increased maintenance. Instead, let’s add the testimonials in front matter and iterate over them with Liquid:
--- layout: default testimonials: - message: We use Justice Law in all our endeavours. They offer an unparalleled service when it comes to running a business. image: "/images/joice.jpeg" name: Joice Carmold - message: Justice Law are the best of the best. Being local, they care about people and have strong ties to the community. image: "/images/peter.jpeg" name: Peter Rottenburg - message: Justice Law were everything we could have hoped for when buying our first home. Highly recommended to all. image: "/images/gibblesto.jpeg" name: D. and G. Gibbleston --- <p>Justice Law is professional representation. Practicing for over 50 years, our team have the knowledge and skills to get you results.</p> <div> {% for testimonial in page.testimonials %} <blockquote> <p>{{ testimonial.message }}</p> <p> <img src="{{ testimonial.image }}" alt="Photo of {{ testimonial.name }}"> {{ testimonial.name }} </p> </blockquote> {% endfor %} </div>
WordPress stores the HTML content, date and other meta data for posts in the database.
In Jekyll, each post is a static file stored in a _posts directory. The file name has the publication date and title for the post — for example, _posts/2016-11-11-real-estate-flipping.md. The source code for a blog post takes this structure:
--- layout: post categories: - Property --- Flipping is a term used primarily in the US to describe purchasing a revenue-generating asset and quickly reselling it for profit. 
We can also use front matter to set categories and tags.
Below the front matter is the body of the post, written in Markdown19. Markdown is a simpler alternative to HTML.
Jekyll allows us to create layouts that inherit from other layouts. You might have noticed this post has a layout of post. The post layout inherits from the default layout and adds a date and title:
In WordPress, custom post types are useful for managing groups of content. For example, you might use custom post types for testimonials, products or real-estate listings.
The about.html page shows profiles of staff members. We could define the meta data for the staff (name, image, phone number, bio) in the front matter, but then we could only reference it on that page. In the future, we want to use the same data to display information about authors on blog posts. A collection enables us to refer to staff members anywhere on the website.
Configuration of our website lives in _config.yml. Here, we set a new collection:
collections: staff_members: output: false
Now we add our staff members. Each staff member is represented in a Markdown file stored in a folder with the collection name; for example, _staff_members/jane-doe.md.
We add the meta data in the front matter and the blurb in the body:
--- name: Jane Doe image: "/images/jane.jpeg" phone: "1234567" --- Jane has 19 years of experience in law, and specialises in property and business.
Similar to posts, we can iterate over the collection in about.html to display each staff member:
--- layout: default --- <ul> {% for member in site.staff_members %} <li> <div><img src="{{ member.image }}" alt="Staff photo for {{ member.name }}"></div> <p>{{ member.name }} - {{ member.phone }}</p> <p>{{ member.content | markdownify }}</p> </li> {% endfor %} </ul>
Some WordPress plugins can be emulated with core Jekyll. Here’s a photo gallery using front matter and Liquid:
--- layout: default images: - image_path: /images/bill.jpg title: Bill - image_path: /images/burt.jpg title: Burt - image_path: /images/gary.jpg title: Gary - image_path: /images/tina.jpg title: Tina - image_path: /images/suzy.jpg title: Suzy --- <ul> {% for image in page.images %} <li><img src="{{ image.image_path }}" alt="{{ image.title }}"/></li> {% endfor %} </ul>
We just need to add our own JavaScript and CSS to complete it.
Jekyll plugins can emulate the functionality of other WordPress plugins. Keep in mind that Jekyll plugins only run while the website is being generated — they don’t add real-time functionality:
One of the major benefits of using a static site generator like Jekyll is the entire site and content can live in Git. At a basic level, Git gives you a history of all the changes on the site. For teams, it opens up all sorts of workflows and approval processes.
That covers the nuts and bolts of creating the website. If you’re curious to see how an entire Jekyll website fits together, have a look at the Justice template47. It’s a free MIT-licensed template for Jekyll. The snippets above are based on this template.
The WordPress CMS is built into the platform, so we would need to set up an account for the client.
With our Jekyll website, we’d link our Git repository to one of the Jekyll GUIs mentioned earlier. One of the nice things about this workflow is that clients changes are committed back to the repository. As developers, we can continue to use local workflows even with non-developers updating the website.
Some Jekyll GUIs offer hosting, while others have a way to output to an Amazon S3 bucket or to GitHub Pages48.
At this point, our Jekyll website is live and editable by the client. If we need to make any changes to the website, we simply push to the repository and it will automatically deploy live.
Now it’s your turn. Plenty of resources are available to help you build your first Jekyll website:
The official Jekyll website49 is a great place to start with in-depth documentation on all of Jekyll’s features.
Jekyll.tips50 has a video tutorial series covering core Jekyll topics.
Have a look at Jekyll templates on GitHub to see how they’re put together: Frisco51 for marketing websites, Scholar52 for documentation and Urban53 for digital agencies.
If you’re migrating, Jekyll has tools to import posts54 from WordPress and WordPress.com websites. After importing, you’ll need to manually migrate or create the layouts, pages, CSS, JavaScript and other assets for the website.
The beauty of Jekyll is in its simplicity. While WordPress can match many of the features of Jekyll, it often comes at the cost of complexity through extra plugins or infrastructure.
Ultimately, it’s about finding the tool that works best for you. I’ve found Jekyll to be a fast and efficient way to build websites. I encourage you to try it out and post your experience in the comments.
No matter whether you are designing a whole design system or just a couple of screens, symbols in Sketch will help you keep your file organized and will save you a lot of time in the long run. In this article, I’ll share with you a few best practices and tricks to help you unleash symbols’ full potential.
But first, a bit of a backstory. I started using Sketch a few years ago, as a replacement for my favorite design software back then, Fireworks1, which had been discontinued2 by Adobe — leaving a whole generation of designers3 broken-hearted. Since my first days of using Sketch, I was very surprised by how easy and straightforward it is to use. I had, once again, found an application focused on user interface (and icon) design — and nothing else.
The apparent lack of features in Sketch, compared to the alternatives full of menus and stacked panels4 that I was used to, was in fact one of its major advantages and helped me to design faster. Among those few features, symbols were the thing that I used very frequently, and still do, practically every day (yes, even on Sundays… you know, a freelancer’s life).
What are symbols? In a nutshell, symbols enable you to use and reuse an element across a project, keeping a master symbol that automatically updates other instances of the symbol when changes are made to it.
This concept is not exactly new (nor is it exclusive to Sketch, to be honest). However, if you design interfaces, then you’ll find it extremely useful, especially when using components as part of a design system8.
In this article, I’ll outline how to make use of symbols in Sketch in order to unleash their full potential, going from the most basic situations to some more advanced use cases. I’ll also include some tips and tricks that I have learned along the way.
Before digging deeper, and in case you are new to Sketch, let me give you a short introduction to how symbols work.
Symbols can be made from almost any elements in Sketch: text objects, shapes, bitmap images, even other symbols (we’ll talk about this later). Inside every symbol (double-click a symbol to enter edit mode), you’ll find one main artboard containing the symbol’s layers. This artboard also defines the symbol’s boundaries.
Usually, symbols are created for those elements in an interface that you expect to reuse later on (such as buttons, list items, tabs, etc.) and that will be spread across different screens, pages and artboards in your designs.
Note: For future reference, keep in mind that “copies” of one symbol are called instances.
The best thing about using symbols (instead of grouped, independent and disconnected objects) is that if at some point you decide to change some property in a particular symbol (for example, the color, shape, text size, dimensions or whatever else you want), you’ll just need to edit the symbol’s master once, and this change will be automatically replicated to all of the master’s instances, wherever they are. I don’t know about you, but I find this super-convenient!
Just like in life itself, it’s fundamental to keep everything in order. Always design as if someone else will later need to open and work with your design file and understand it without your help! This also applies to the way you name symbols — naming should meet certain criteria.
One recommendation is to use a slash (/) in the symbol’s name. Sketch will automatically create a category with the part before the slash, and will name and place the symbol inside it, using the part of the name following the slash. For example, if you have two symbols named “Button/Primary” and “Button/Secondary,” here is how they will look like when you try to insert them from the toolbar:
13 Naming symbols properly can help you create a neat hierarchy system, which will help you find symbols faster. (View large version14)
You can repeat this many times to have several symbols under the same root, grouped by similar logic, making them easier to find. And if your “tree” grows too big, take a moment to reconsider your naming system and see if there’s any possible way to optimize it and make it more manageable.
There are many different conventions for how symbols should be named, perhaps one convention for every designer out there. Personally, I prefer not to use names that refer to the visual properties of the elements — for example, “Red Button” would be a bad choice in my opinion because if the color of the button changes later on for some reason, the name of the symbol will become incorrect. Instead, I try to differentiate the symbol’s function and state (such as “Primary/Disabled”).
In any case, just be consistent and find something that works for both you and your team, then stick to it; don’t switch the naming system according to the case! This also applies to layers inside symbols: some designers even use emojis15 to mark which of them are meant to be editable (for example, by adding a pencil emoji to the name). To do this, press Control + Command + Space to open a dialog to select emojis16.
17 Adding an emoji to a layer’s name is an option when you want to highlight its purpose.
Note: Regarding symbols’ names, bear in mind that instances will take their names from the master symbol, but you can change them afterwards to whatever you want. This way, instances of the same symbol can have different names from each other.
When you create a symbol, Sketch asks whether you want to send it to the symbols page18. My advice is to check this box, even if after a while (and a few symbols later) this dedicated page turns into a mess. (Sketch places one symbol next to the other as they are being created, and when you delete a symbol, you’ll notice the blank space left in its spot.)
Instead, what I do to sort this out is to create my own symbols page (which is just a regular page, which I would usually name “Symbols”) where I can arrange symbol instances in the order I want and, thus, ignore the official symbols page.
19 The symbols page (left) can easily become disorganized, which is why I prefer to have my own separate page (right) where I can organize symbols’ instances in the way I think best. (View large version20)
This way, I can create artboards that follow categories (such as lists, buttons, inputs and so on) and place symbols in a way that I find convenient and that makes sense to me. You’ll still need to invest some time to update this page from time to time, but once it is created, it will make everything much easier and you’ll be able to build a new screen in no time.
Note: If you prefer to use the symbols page instead, there’s the Symbol Organizer plugin21, which could help you keep everything arranged.
Replacing an existing symbol with another is easy. Just select the symbol and choose “Replace with” from the contextual menu that appears when you right-click over the symbol instance. Then, select the new symbol that you want to use. Keep in mind that the new symbol will keep the same size and position as its predecessor; you can fix this by selecting “Set to original size” from the same contextual menu.
Once you’ve made a symbol, you can detach it to recover the elements that form it as a group. To do this, just select “Detach from symbol” in the same contextual menu that I mentioned earlier.
Symbols, like other elements, can also be exported as bitmap images. To do this, you’ll need to mark elements as exportable. (Select the symbol instance, and then choose “Make Exportable” at the bottom of the Inspector.)
The problem that I found during this process is that if the symbol has some padding (for example, if the shapes inside are smaller than the symbol’s total size), when doing the export, Sketch will omit the blank space and will just create an image with the visible content only.
23 Exporting the symbol directly will create an image that contains only the shapes (left). Make use of slices (right) to match the symbol size before exporting. (View large version24)
One way to work this around is by using a slice25. When creating the slice, place it over the instance and make sure it matches the size of the instance’s boundaries (width and height); then, select the slice and use the exporting options as needed.
Side note: This same trick also applies to other tools, such as Zeplin26.
In this world full of screens with multiple sizes and aspect ratios, it’s important to make sure your design adapts to many different scenarios. This is easier to accomplish if you don’t have to design everything from scratch every time, by reusing elements (or symbols, as I’m sure you’ve already guessed).
29 A variety of resizing options are available in Sketch. These are very useful when you are looking to adapt your symbols to different dimensions.
This is where the resizing options in symbols come in handy, helping you to use the same element with different widths and heights with no hassle: If you resize just one instance by selecting it, this won’t affect the other instances. (But remember that resizing options are applied to individual layers inside the master symbol, not to the instance itself. So, even while you can adjust sizes individually from instance to instance, elements inside will always maintain the same behavior.)
Note: The options outlined below apply not only to symbols, but to groups as well. Behaviors are not always predictable, so chances are that you’ll have to play around and explore a bit before finding what you need, combining one or two different settings in most cases.
30 In this example you can see how the yellow shape will remain one third of the total width of the symbol, no matter what the total width of the symbol is.
When the Stretch option is used, a shape that has specified, let’s say, 50% of the symbol’s total width will keep this same relationship when the instance is extended vertically or horizontally. This is the default behavior.
31 The yellow shape has the “Pin to Corner” option applied to it, so it sticks to the closest corner.
“Pin to Corner” will (as the name suggests) pin an element to the nearest corner, and the element will not resize, keeping the same distance to this corner. Keep in mind that if the object is centered (with equal spacing from both sides), it won’t know which one is the nearest corner, so it’ll stay in the middle.
32 The yellow shape has the “Resize Object” option applied to it. You can better understand what is happening by also looking at the pink squares, which are for reference only.
When “Resize Object” is used, elements will grow while keeping an equal (or fixed) spacing from the sides.
If you have resized your symbol but aren’t satisfied with the result, you can always go back to the beginning by choosing “Set to original size” from the contextual menu.
Keep in mind that symbols have dedicated artboards, and these will define the symbols’ boundaries (even when shapes inside overflow on them). You can make the symbol’s artboard the same size as of its contents by selecting it and choosing “Resize to fit” from the Inspector.
In the width and height input fields in the Inspector, you can use operators to change values. For instance, you can use 100*2 to set an element’s dimensions to 200 pixels. Other operators are + (add), - (subtract) and / (divide).
Besides mathematical operators, in the same input fields you can also use L to scale an object from the left (this is the default), R to scale it from the right, T to scale it from the top (this is the default), B to scale it from the bottom, and C and M to scale it from the center or middle.
34 When you resize a rectangle using its upper-right corner as an anchor, the shape will grow out of its left and bottom sides.
For example, if you have a shape that has a width of 200 pixels and want to resize it so that it scales from the right to the left side, you can use something like 300r in the width input field.
What could be better than one symbol? Perhaps a symbol with another one inside it!
This feature is kind of new in Sketch, and it gives you a lot of possibilities when combining symbols together. You can place one symbol on top of another, select both, and then create a new symbol that contains the two instances. You can repeat this as much as you’d like. Be moderate, though, or else you’ll find yourself digging into levels and levels of nested symbols, one inside another. This could make maintenance much harder and could also be a symptom of bigger organizational problems.
Nesting symbols can be especially useful when you need to create variations of one symbol. For example, you could follow a process like this:
Pick up one symbol that will serve as a base. (this symbol will remain the same in all cases.)
Overlap it with other symbols (such as icons or badges), which could be there or not, depending on the case.
Finally, create another symbol with the resulting design.
In the image below, you can see that all rows share the same characteristics (they have the same size, text properties and amount of padding on the left), so I created a base symbol that contains only these elements (i.e. elements that will be shared with the other symbols). Using this symbol as a starting point, I then created some overlapping elements that are different, saving the result in each case as a different symbol; so, all of the symbols under “Variations” are actually different symbols.
36 An example of different symbols created from a common base, with variations in each case.
But you don’t — necessarily — need to create a new symbol for every state of the row. There may be a simpler way: using overrides.
If you had to create a lot of different symbols just because one part of their content changes, you’d probably go nuts. One of the main purposes of symbols is precisely to have to design as little as possible and to have fewer elements — and, therefore, more control over them. Enter nested overrides!
37 This navigation bar is a symbol that contains the inactive tabs (which are symbols as well). Active tabs are symbols created separately; they override the inactive tabs when necessary.
One practical example of this workflow could be designing a tab bar with different states. In this case, the main symbol with the inactive tabs would act as the base, and then there would be a different symbol for each one of the highlighted tabs. Just choose the one that you want from the “Overrides” options in the Inspector.
Note: For this technique to work, keep in mind that the inactive tabs inside the main symbol (the navigation bar) need to be symbols as well. Also, be sure that all symbols (both inactive and active ones) have the exact same dimensions (width, height). Otherwise, they won’t appear as available options in the “Overrides” dropdown menu.
Let’s look at another use case. If you have multiple buttons in a design but with different text labels on them, then the Overrides option will enable you to change the text value (not the font family or font size — you have to modify those inside the symbol itself, when editing the symbol master), without having to create a new symbol each time. This is as easy to do as selecting the instance and changing the text content in the Inspector.
Overrides apply not only to text; you can also use them for bitmap images and even for other symbols, as mentioned before. This way, you can have several instances of a symbol, with a different image in each one of them — and all of this without having to modify the symbol’s master.
38 Let me show you how overriding images works with something that I usually do when designing.
There are cases when I don’t want to have any particular image as part of a symbol’s master. So, what I usually do is to create an empty PNG file with no visible content, create a shape, and use this image as a pattern fill (you can find this option in the “Fill Options” when selecting a shape). Then, when doing the symbol overriding, I just replace this transparent image with the one that I want in each case!
To get the most out of this practice, I also use a layering system with an icon or element that acts as a placeholder underneath the image and that will be visible only if I keep the original transparent bitmap. One benefit of doing this is that I can simulate this empty state that will appear when images are loading in the finished product, something that I consider necessary to design anyway.
One of the reasons why being organized is a good idea is because the way you name and order layers will affect the way they are displayed in the “Overrides” panel. The labels to the left of the input fields in the Inspector will respect the name and order you’ve previously defined inside the symbol itself, so you’d better pay attention to this order if you want to have a more efficient workflow.
You can replace a nested symbol with another symbol only if the new symbol has the exact same width and height as the current element.
Tip 3: Displacing Elements Depending on Text Length Link
When changing the text’s value in the Overrides options, you can make an element move as needed when the one to its left is longer (see the following illustration).
39 Top (first field): This shows the original symbol. Bottom (second field): This illustrates how the “Optional” label moves to the right because of the previous label’s length (in this case, “Phone Number”).
The secondary text or shape necessarily needs to be to the right of the text for this to work. Also, both elements should have no more than 20 pixels of distance between them (see the “Further Reading” below).
A symbol can look a bit messy because of the options in the Overrides section. If you don’t want an element inside it to be able to be overridden, just lock or hide this layer and it won’t appear in the list.
There’s one way to quickly make a text element disappear in an instance, by using overrides. To do this, just set the text value to a blank space, pressing the space bar and the return key in the Overrides options.
If you have bitmap images inside a symbol, they can be changed by others using the options in the Overrides section. It’s also possible to recover the original image (the one that forms part of the editable symbol) by choosing “Remove image override” — just right-click over the image box next to “Choose Image” in the Inspector.
“Hacking the Button in Sketch40,” Aleksandr Pasevin41 Presents a simple hack to keep an icon to the left of the text (instead of to the right, which is the normal behavior), in just a couple of simple steps.
One good thing about Sketch is that when it falls short of a feature, there’s usually a plugin to make up for it. And some of them work especially well with symbols, making them even more powerful! Some of these plugins have been mentioned, but in case you missed any of them, here’s a list with some additions.
50 Sketch Runner lets you quickly insert symbols by navigating a list, and it also shows a preview in the right panel. (View large version51)
Among its many other features, the Sketch Runner52 plugin will help you easily insert symbols in a document using just a combination of keys. The “go to” option is very useful for jumping right to a particular symbol — very useful if your project has a lot of them and if it’s difficult to find symbols using other means.
If you are working with a team, InVision Craft Library53 will make it easy to create a shared library with assets that everybody can use, allowing you to sync changes when you need to update a symbol, so that you are always sure you’re using the symbols’s latest version.
Automate54 is very powerful and will likely make your work more efficient. Options for managing symbols include ones to remove unused symbols, to select all instances of a symbol, and much more.
With Symbol Organizer56, organize your symbols page alphabetically (including the layers list) and into separate groups determined by your symbol names.
Auto Layout57 integrates seamlessly in Sketch and enables defining and viewing different iPhone and iPad sizes including portrait and landscape. It also supports more advanced features, such as stacks (a special type of group that defines the layouts of its child layers), and presets for both Android and iOS. Look at their “Examples” page58 for more information.
Note: These are only some of the plugins that I think might be most helpful to you, but there are many others. To know more, just visit Sketch’s official plugin page59 or the Sketch App Sources60 website regularly.
Sketch symbols are constantly evolving, so we can expect further improvements that will make them even more valuable and relevant. However, if I had to name just one thing that I would like them to have, that would be the possibility to have shared symbols’ libraries, something like Figma is doing61. This could be extremely useful, especially for team work, when several designers working on the same project need to pick elements from a primary, always up-to-date document stored in the cloud.
(Note: Regarding this feature, I’m aware that Sketch’s team is working on it, so hopefully we’ll see it soon. The more open format in version 4362 is probably laying the groundwork for this feature. In any case, I’m looking forward to it, because this could be a game-changer in many designer workflows.)
Truth be told, there are currently some plugins that help you accomplish more or less the same behavior mentioned above, but I always find it more reliable when they are made a part of Sketch’s core functionality — which ensures that the feature will keep working when the software is updated to the next version.
I’m aware that there are many more techniques and tricks. The way one works tends to be kind of personal sometimes, and there’s no single right way to do something. Here, I’ve shared the techniques that I think are reliable, interesting and don’t require much hacking. That’s why some techniques were left out of this article.
I hope this was a useful read! If it was, then symbols will probably become the backbone of your designs, and you’ll use them quite often. Feel free to share your thoughts and other tips and tricks in the comments below. You can also always reach me on Twitter63 if you need help!
Today, CSS preprocessors are a standard for web development. One of the main advantages of preprocessors is that they enable you to use variables. This helps you to avoid copying and pasting code, and it simplifies development and refactoring.
We use preprocessors to store colors, font preferences, layout details — mostly everything we use in CSS.
But preprocessor variables have some limitations:
You cannot change them dynamically.
They are not aware of the DOM’s structure.
They cannot be read or changed from JavaScript.
As a silver bullet for these and other problems, the community invented CSS custom properties. Essentially, these look and work like CSS variables, and the way they work is reflected in their name.
Custom properties are opening new horizons for web development.
The usual problem when you start with a new preprocessor or framework is that you have to learn a new syntax.
Each preprocessor requires a different way of declaring variables. Usually, it starts with a reserved symbol — for example, $ in Sass and @ in LESS.
CSS custom properties have gone the same way and use -- to introduce a declaration. But the good thing here is that you can learn this syntax once and reuse it across browsers!
You may ask, “Why not reuse an existing syntax?”
There is a reason5. In short, it’s to provide a way for custom properties to be used in any preprocessor. This way, we can provide and use custom properties, and our preprocessor will not compile them, so the properties will go directly to the outputted CSS. And, you can reuse preprocessor variables in the native ones, but I will describe that later.
(Regarding the name: Because their ideas and purposes are very similar, sometimes custom properties are called the CSS variables, although the correct name is CSS custom properties, and reading further, you will understand why this name describes them best.)
So, to declare a variable instead of a usual CSS property such as color or padding, just provide a custom-named property that starts with --:
In case you are not sure what :root6 matches, in HTML it’s the same as html but with a higher specificity.
As with other CSS properties, custom ones cascade in the same way and are dynamic. This means they can be changed at any moment and the change is processed accordingly by the browser.
To use a variable, you have to use the var() CSS function and provide the name of the property inside:
The var() function is a handy way to provide a default value. You might do this if you are not sure whether a custom property has been defined and want to provide a value to be used as a fallback. This can be done easily by passing the second parameter to the function:
.box{ --box-color:#4d4e53; --box-padding: 0 10px; /* 10px is used because --box-margin is not defined. */ margin: var(--box-margin, 10px); }
As you might expect, you can reuse other variables to declare new ones:
.box{ /* The --main-padding variable is used if --box-padding is not defined. */ padding: var(--box-padding, var(--main-padding)); --box-text: 'This is my box'; /* Equal to --box-highlight-text:'This is my box with highlight'; */ --box-highlight-text: var(--box-text)' with highlight'; }
As we got accustomed to with preprocessors and other languages, we want to be able to use basic operators when working with variables. For this, CSS provides a calc() function, which makes the browser recalculate an expression after any change has been made to the value of a custom property:
Before talking about CSS custom property scopes, let’s recall JavaScript and preprocessor scopes, to better understand the differences.
We know that with, for example, JavaScript variables (var), a scope is limited to the functions.
We have a similar situation with let and const, but they are block-scope local variables.
A closure in JavaScript is a function that has access to the outer (enclosing) function’s variables — the scope chain. The closure has three scope chains, and it has access to the following:
its own scope (i.e. variables defined between its braces),
This means that, in Sass, the variable’s scopes fully depend on the code’s structure.
However, CSS custom properties are inherited by default, and like other CSS properties, they cascade.
You also cannot have a global variable that declares a custom property outside of a selector — that’s not valid CSS. The global scope for CSS custom properties is actually the :root scope, whereupon the property is available globally.
Let’s use our syntax knowledge and adapt the Sass example to HTML and CSS. We’ll create a demo using native CSS custom properties. First, the HTML:
global <div> enclosing <div> closure </div> </div>
That’s the first huge difference: If you reassign a custom property’s value, the browser will recalculate all variables and calc() expressions where it’s applied.
Preprocessors Are Not Aware of the DOM’s Structure Link
Suppose we wanted to use the default font-size for the block, except where the highlighted class is present.
.highlighted { --highlighted-size: 30px; } .default { --default-size: 10px; /* Use default-size, except when highlighted-size is provided. */ font-size: var(--highlighted-size, var(--default-size)); }
Because the second HTML element with the default class carries the highlighted class, properties from the highlighted class will be applied to that element.
In this case, it means that --highlighted-size: 30px; will be applied, which in turn will make the font-size property being assigned use the --highlighted-size.
This happens because all Sass calculations and processing happen at compilation time, and of course, it doesn’t know anything about the DOM’s structure, relying fully on the code’s structure.
As you can see, custom properties have the advantages of variables scoping and add the usual cascading of CSS properties, being aware of the DOM’s structure and following the same rules as other CSS properties.
The second takeaway is that CSS custom properties are aware of the DOM’s structure and are dynamic.
CSS custom properties are subject to the same rules as the usual CSS custom properties. This means you can assign any of the common CSS keywords to them:
inherit
This CSS keyword applies the value of the element’s parent.
initial
This applies the initial value as defined in the CSS specification (an empty value, or nothing in some cases of CSS custom properties).
unset
This applies the inherited value if a property is normally inherited (as in the case of custom properties) or the initial value if the property is normally not inherited.
revert
This resets the property to the default value established by the user agent’s style sheet (an empty value in the case of CSS custom properties).
Let’s consider another case. Suppose you want to build a component and want to be sure that no other styles or custom properties are applied to it inadvertently (a modular CSS solution would usually be used for styles in such a case).
But now there is another way: to use the all CSS property26. This shorthand resets all CSS properties.
Together with CSS keywords, we can do the following:
The name of these CSS variables is “custom properties,” so why not to use them to emulate non-existent properties?
There are many of them: translateX/Y/Z, background-repeat-x/y (still not cross-browser compatible), box-shadow-color.
Let’s try to make the last one work. In our example, let’s change the box-shadow’s color on hover. We just want to follow the DRY rule (don’t repeat yourself), so instead of repeating box-shadow’s entire value in the :hover section, we’ll just change its color. Custom properties to the rescue:
One of the most common use cases of custom properties is for color themes in applications. Custom properties were created to solve just this kind of problem. So, let’s provide a simple color theme for a component (the same steps could be followed for an application).
This has everything we need. With it, we can override the color variables to the inverted values and apply them when needed. We could, for example, add the global inverted HTML class (to, say, the body element) and change the colors when it’s applied:
This behavior cannot be achieved in a CSS preprocessor without the overhead of duplicating code. With a preprocessor, you would always need to override the actual values and rules, which always results in additional CSS.
With CSS custom properties, the solution is as clean as possible, and copying and pasting is avoided, because only the values of the variables are redefined.
Previously, to send data from CSS to JavaScript, we often had to resort to tricks37, writing CSS values via plain JSON in the CSS output and then reading it from the JavaScript.
Now, we can easily interact with CSS variables from JavaScript, reading and writing to them using the well-known .getPropertyValue() and .setProperty() methods, which are used for the usual CSS properties:
/** * Gives a CSS custom property value applied at the element * element {Element} * varName {String} without '--' * * For example: * readCssVar(document.querySelector('.box'), 'color'); */ function readCssVar(element, varName){ const elementStyles = getComputedStyle(element); return elementStyles.getPropertyValue(`--${varName}`).trim(); } /** * Writes a CSS custom property value at the element * element {Element} * varName {String} without '--' * * For example: * readCssVar(document.querySelector('.box'), 'color', 'white'); */ function writeCssVar(element, varName, value){ return element.style.setProperty(`--${varName}`, value); }
Let’s assume we have a list of media-query values:
To show how to assign custom properties from JavaScript, I’ve created an interactive 3D CSS cube demo that responds to user actions.
It’s not very hard. We just need to add a simple background, and then place five cube faces with the relevant values for the transform property: translateZ(), translateY(), rotateX() and rotateY().
To provide the right perspective, I added the following to the page wrapper:
The only thing missing is the interactivity. The demo should change the X and Y viewing angles (--rotateX and --rotateY) when the mouse moves and should zoom in and out when the mouse scrolls (--translateZ).
Essentially, we’ve just changed the CSS custom properties’ values. Everything else (the rotating and zooming in and out) is done by CSS.
Tip: One of the easiest ways to debug a CSS custom property value is just to show its contents in CSS generated content (which works in simple cases, such as with strings), so that the browser will automatically show the current applied value:
You can check it in the plain CSS demo42 (no HTML or JavaScript). (Resize the window to see the browser reflect the changed CSS custom property value automatically.)
This means that, you can start using them natively.
If you need to support older browsers, you can learn the syntax and usage examples and consider possible ways of switching or using CSS and preprocessor variables in parallel.
Of course, we need to be able to detect support in both CSS and JavaScript in order to provide fallbacks or enhancements.
This is quite easy. For CSS, you can use a @supports condition46 with a dummy feature query:
@supports ( (--a: 0)) { /* supported */ } @supports ( not (--a: 0)) { /* not supported */ }
In JavaScript, you can use the same dummy custom property with the CSS.supports() static method:
As we saw, CSS custom properties are still not available in every browser. Knowing this, you can progressively enhance your application by checking if they are supported.
For instance, you could generate two main CSS files: one with CSS custom properties and a second without them, in which the properties are inlined (we will discuss ways to do this shortly).
Load the second one by default. Then, just do a check in JavaScript and switch to the enhanced version if custom properties are supported:
<!-- HTML --> <link href="without-css-custom-properties.css" rel="stylesheet" type="text/css" media="all" />
// JavaScript if(isSupported){ removeCss('without-css-custom-properties.css'); loadCss('css-custom-properties.css'); // + conditionally apply some application enhancements // using the custom properties }
This is just an example. As you’ll see below, there are better options.
One advantage of this method of manually checking in the code whether custom properties are supported is that it works and we can do it right now (don’t forget that we have switched to Sass):
This method does have many cons, not least of which are that the code gets complicated, and copying and pasting become quite hard to maintain.
2. Use a Plugin That Automatically Processes the Resulting CSS Link
The PostCSS ecosystem provides dozens of plugins today. A couple of them process custom properties (inline values) in the resulting CSS output and make them work, assuming you provide only global variables (i.e. you only declare or change CSS custom properties inside the :root selector(s)), so their values can be easily inlined.
This plugin offers several pros: It makes the syntax work; it is compatible with all of PostCSS’ infrastructure; and it doesn’t require much configuration.
There are cons, however. The plugin requires you to use CSS custom properties, so you don’t have a path to prepare your project for a switch from Sass variables. Also, you won’t have much control over the transformation, because it’s done after the Sass is compiled to CSS. Finally, the plugin doesn’t provide much debugging information.
This gives you a way to control all of the CSS output from one place (from Sass) and start getting familiar with the syntax. Plus, you can reuse Sass variables and logic with the mixin.
When all of the browsers you want to support work with CSS variables, then all you have to do is add this:
$css-vars-use-native: true;
Instead of aligning the variable properties in the resulting CSS, the mixin will start registering custom properties, and the var() instances will go to the resulting CSS without any transformations. This means you’ll have fully switched to CSS custom properties and will have all of the advantages we discussed.
If you want to turn on the useful debugging information, add the following:
$css-vars-debug-log: true;
This will give you:
a log when a variable was not assigned but was used;
a log when a variable is reassigned;
information when a variable is not defined but a default value gets passed that is used instead.
Now you know more about CSS custom properties, including their syntax, their advantages, good usage examples and how to interact with them from JavaScript.
You have learned how to detect whether they are supported, how they are different from CSS preprocessor variables, and how to start using native CSS variables until they are supported across browsers.
This is the right time to start using CSS custom properties and to prepare for their native support in browsers.
Looking at recent discussions, I feel that more and more people are starting to think about ethically and morally correct work. Many of us keep asking themselves if their work is meaningful or if it matters at all. But in a well-functioning society, we need a variety of things to live a good life. The people writing novels that delight us are just as important as those who fight for our civil rights.
It’s important that we have people building services that ease other people’s lives and it’s time to set our sense of urgency right again. Once we start to value other people’s work, the view we have on our own work will start to change, too. As we rely on book authors, for example, other people rely on us to be able to buy the books via a nice, fast and reliable web service.
Good news if you’re using PostgreSQL: The upcoming PostgreSQL 10 offers some great new features5. It’ll support logical replication in addition to the already existing logical decoding, up to 4x faster parallel query, SCRAM Authentication, and a lot of other useful things.
Alexis Deveria creates the amazing project caniuse.com6, a site that we all use a lot. Now he accepts donations, and in return, you can also get an ad-free experience on the site. If you rely on caniuse.com for your work, consider showing your appreciation for the hard work that the author puts into it by giving something back7.
With the new Windows 10 Creators Update8, Edge 15 went live. It comes with a new tab management interface, a book reader mode, and better energy efficiency. But even more interesting for us are the implementation of the Web Payment Request API, CSS Custom Property support, Brotli support, WebRTC, async/await, and Intersection Observer.
André Staltz shares his most valuable piece of advice to be a better programmer: “Gain a deeper understanding of the system,” and he has strong points in his article9 that reinforce this.
There’s a new DNS resource record: CAA. The Certificate Authority Authorization Record10 lets you specify which certificate authority is allowed to issue a certificate for your domain. From September 2017 on, CAs are required to check against these records, so you should consider adding that record to your DNS records as soon as possible.
Cassie Marketos shares her concerns and discoveries about “What Makes Work Meaningful21.” In search of the answer to the common question if what we do matters at all, this article reveals some thoughts that set our sense of urgency right again.
Josh Clark on why the smart algorithm systems that power Google, Siri, Alexa and other “intelligent” AI services should know when they’re not smart enough23 and indicate that to users.
Microplastics are everywhere. They’re used in most creams, shower gels and a lot of other products we use every day. Scientists now found microplastics in commercial salts24 from several countries, indicating how badly our seas are polluted with these particles. As we’re eating salt, this has a direct effect on our health, and it’s only stoppable if we achieve to not have microplastic particles in everyday products anymore.
Last but not least, if you’re in Europe or Germany, how about joining the awesome CSSconf EU in Berlin on May, 5th25? There are still tickets available. I’ll be around at the sold-out beyondtellerrand in Düsseldorf again, and I’d love to meet you there. If you don’t have a ticket, maybe join one of the side events26? Or consider the Material Conference27 which will take place on August 17th in Iceland, a lovely island, and I’m sure the event will be great as well.
For the past few months, I’ve been building a software-as-a-service (SaaS) application, and throughout the development process I’ve realized what a powerful tool Slack (or team chat in general) can be to monitor user and application behavior. After a bit of integration, it’s provided a real-time view into our application that previously didn’t exist, and it’s been so invaluable that I couldn’t help but write up this show-and-tell.
It all started with a visit to a small startup in Denver, Colorado. During my visit, I started hearing a subtle and enchanting “ding” in the corner of the office every few minutes. When I went to investigate this strange noise, I found a service bell hooked up to a Raspberry Pi, with a tiny metal hammer connected to the circuit board. As it turned out, the Pi was receiving messages from the team’s server, and it swung that little hammer at the bell every time a new customer signed up.
I always thought that was a great team motivator, and it got me thinking of how I could use team chat to achieve a similar experience.
Because we were already using Slack for team chat, and because it has a beautifully documented API1, it was an obvious choice for the experiment.
First, we had to obtain a “webhook URL” from Slack in order to programmatically post messages to our Slack channel.
6 Follow the steps above to obtain a webhook URL from Slack (View large version7)
Now that we had a webhook URL, it was time to integrate Slack messages into our Node.js application. To do this, I found a handy Node.js module named node-slack8.
First, we installed the Node.js module:
npm install node-slack --save
Now, we could send Slack messages to our channel of choice with a few lines of code.
// dependency setup var Slack = require('node-slack'); var hook_url = 'hook_url_goes_here'; var slack = new Slack(hook_url); // send a test Slack message slack.send({ text: ':rocket: Nice job, I'm all set up!', channel: '#test', username: 'MyApp Bot' });
(You can find similar Slack integration packages for Ruby9, Python10 and just about any other language.)
When executed, this code produced the following message in our #test Slack channel:
The code above is minimal, but it’s specific to the Slack API and the node-slack module. I didn’t want to be locked into any particular messaging service, so I created a generic Node.js module function to execute the service-specific code:
// Messenger.js // dependency setup var hook_url = my_hook_url; var Slack = require('node-slack'); var slack = new Slack(hook_url); module.exports = { sendMessage: function(message, channel, username) { if (!message){ console.log('Error: No message sent. You must define a message.') } else { // set defaults if username or channel is not passed in var channel = (typeof channel !== 'undefined') ? channel : "#general"; var username = (typeof username !== 'undefined') ? username : "MyApp"; // send the Slack message slack.send({ text: message, channel: channel, username: username }); return; } } };
Now we can use this module anywhere in the application with two lines of code, and if we ever decide to send messages to another service in the future, we can easily swap that out in Messenger.js.
var messenger = require('./utilities/messenger'); messenger.sendMessage(':rocket: Nice job, I'm all set up!', '#test');
Now that we had the basics set up, we were ready to start firing off messages from within the application.
The first order of business was to achieve service-bell parity. I located the success callback of the user registration function, and I added this code:
messenger.sendMessage('New user registration! ' + user.email);
Now, when someone registered, we’d get this message:
As my curiosity grew with each ding, I began to wonder things like, What if there was a failure to create a new user? What if a user registered, logged in but didn’t complete the onboarding process? What is the result of our scheduled tasks? Now that the groundwork was in place, answering these questions was a piece of cake.
Monitor Exceptions and Critical Errors on Back End Link
One of the most important errors we wanted to know about was if there was a failure to create a new user. All we had to do was find the error callback in the user registration function, and add this code:
messenger.sendMessage(':x: Error While adding a new user ' + formData.email + ' to the DB. Registration aborted!' + error.code + ' ' + error.message);
Now we knew instantly when registrations failed, why they failed and, more importantly, who they failed for:
There were all kinds of interesting places where we could send messages (pretty much anywhere with an error callback). One of those places was this generic catch-all error function:
This code helped us to uncover what a request looks like for unhanded exceptions. By looking at the request that triggered these errors, we could track down the root causes and fix them until there were no more generic errors.
With all of these error notifications in place, we now had comfort in knowing that if something major failed in the app, we would know about it instantly.
Next, I wanted to send a notification when a financial event happens in the application. Because our SaaS product integrates with Stripe, we created a webhook endpoint that gets pinged from Stripe when people upgrade their plan, downgrade their plan, add payment info, change payment info and many other events related to subscription payments, all of which are sent to Slack:
There were a few cases on the front end where we wanted to understand user behavior in ways that the back end couldn’t provide, so we created an endpoint to send Slack messages directly from the front end. Because our Slack webhook URL is protected behind a POST endpoint, it was a minimal risk to expose sending Slack messages to our team via an endpoint.
With the endpoint in place, we could now fire off Slack messages with a simple AngularJS $http.post call:
// send Slack notification from the front end var message = ":warning: Slack disconnected by " + $scope.user.username; $http.post('/endpoint', message);
This helps us to answer important questions about the business: Are people registering and adding a domain name? Are they not? If someone is, is it for a really high-profile domain whose owner we would want to reach out to personally soon after they’ve added it. We can now tap into this:
At one point, we saw a pattern of people adding a domain, removing it, then readding it within a few minutes, which clued us into an obscure bug that we probably would never have discovered otherwise.
There are also signals that a user is unhappy with the service, and these are valuable to know about. Did someone remove a domain name? Did they disconnect Slack?
One of the most interesting things to see in Slack is the result of scheduled tasks. Our SaaS product runs tasks to notify people about their website’s performance (our core service), to send transactional emails, to clean up the database and a few other things. The firing and results of these tasks sends a message to Slack:
Now we know when a task function fires, what the result of that function is (in this case, it sends out several emails) and whether it fails for any reason.
The case study above is a practical example of what we did to monitor the GoFaster.io22 application and service. It has worked fantastic for us, but how would this concept scale to large applications that send hundreds, maybe even thousands, of messages per day? As you can imagine, this would quickly turn into a “Slackbot who cried wolf” situation, and the value would get lost in the noise.
Some notifications are more important than others, and importance will vary depending on the employee and their role. For example, software development and IT operations (DevOps) folk might only care about the server messages, whereas customer service folk would care most about what’s going on with users.
Luckily, Slack has a great solution to this problem: channels.
Channels can be created by anyone, made public or private to your organization, and shared with anyone. Once you’ve subscribed to a channel, you can control how that channel’s activities alert you. Does a new message in the channel ding every time? Does it alert your phone, too? Does it only bold the channel? All of this can be controlled for each channel by each team member to suit their needs.
Putting this idea into practice, here’s how a larger organization might organize monitor-based notifications in Slack via channels:
Having built on this idea for a few months and digested the results, we’ve found it to be an invaluable extension of our application. Without it, we would feel out of touch with what is going on with the service and would have to manually hunt down the same information via the dashboard, or database queries would be a chore.
Every application and user base is different, which means that this concept cannot be built into a service and offered to the masses. In order to be valuable, it requires a small up-front investment of time and resources to deeply integrate in your application. Once it’s up and running, the investment will pay off in the form of your team’s connectedness to your application and its users.
In conclusion, here’s a recap of the benefits of using team chat to monitor your application:
Gain a Fresh Perspective on User and Server Behavior Link
Having a real-time live feed of the metrics that matter most to you and your business will keep you closely connected to what users are doing and how the server is responding.
You will be able to react faster than ever before. You will know about failures at the same time your users do. You can immediately react to that failing endpoint, lost database connection or DDoS attack.
Reach out to that customer who has just disabled their account to offer them a discount, give personal thanks to customers who have upgraded, or just follow up with people to understand their intentions. When you know what users are doing and when they are doing it, you can easily find out why.
Team Connectedness to the Application Will Make You More Efficient Link
When your team is on the same page with the application, collaboration can center on solving problems as they arise, rather than on trying to figure out what happened, where it happened or who it happened to.
Notifications and Channels Can Scale With Your Application Link
As your application and team grow, so will your monitoring needs. Slack does a great job of giving you all of the permission and notification controls necessary to ensure that the right information gets to the right people.
By logging a user name in your Slack messages, you can track every error, success message or event that a user has generated while interacting with your application simply by searching for their user name in Slack. Just know that, with a free Slack account, this is limited to the last 10,000 messages.
I hope you’ve found this concept to be useful, and I’d love to hear other stories of teams that have implemented similar forms of monitoring, or just other interesting ways to use and build on it.
Big news from Google: Within a few months, the infamous search engine will divide its index1 to give users better and fresher content. The long-term plan is to make the mobile search index the primary one. Why does this matter for e-commerce website owners?
Well, it will enable Google to run its ranking algorithm differently for purely mobile content. This means that mobile content won’t be extracted from desktop content to determine mobile rankings. That’s definitely something that retailers can leverage, thanks to AMP. This article outlines how to get started with AMP and how to gain an edge over the competition with your e-commerce website.
So, how do online retailers go about leveraging this big Google announcement? With AMP content! AMP (Accelerated Mobile Pages) just celebrated its one-year anniversary. It is an open-source project supported by Google that aims to reduce page-loading times on mobile. AMP pages are similar to HTML pages, with a few exceptions: Some tags are different, some rules are new, and there are plenty of restrictions on the use of JavaScript and CSS.
AMP pages get their own special carousel in Google mobile search results. No official statement has been made yet about whether these AMP pages will be getting an SEO boost.
While initially geared to blogs and news websites, AMP has introduced components that make it easy to adapt to an e-commerce website. To date, more than 150 million AMP documents are in Google’s index, with over 4 million being added every week. AMP isn’t meant purely for mobile traffic; it renders well on mobile, tablet and desktop. The AMP project’s website9 is actually coded in AMP HTML, in case you are curious to see what AMP looks like on a desktop. eBay was one of the most notable early adopters in the e-commerce realm; by July 2016, it took more than 8 million product pages live in AMP format and plans on going further.
Google is touting a reduction of 15 to 85% in page-loading time on mobile. The main advantage of AMP for retailers is that slow loading times kill conversions. Selling products to people when they want them makes a huge difference to a business’ bottom line. Many shoppers will go to a competitor’s website if yours is too slow to load. Put that in a mobile context, and a slow loading time means losing 40% of visitors — potential customers who will take their dollars elsewhere.
In brick and mortar stores, shop fronts are a big deal in attracting customers. It’s the same online, except that your storefront is supported by the speed of your customers’ Internet connection and the visibility you get on various channels (such as search engines, social media and email). Visibility is another way retailers can leverage AMP. Visibility is also a major element of the AMP equation. This is especially true in countries with limited mobile broadband speed10. And before you think this particular challenge is exclusive to developing nations, keep in mind that the US is not ranked in the top 10 countries in mobile broadband speed.
AMP pages feel like they load blazingly fast. Here’s a comparison:
Non-AMP page loading
AMP page loading
Mobile-Friendly Is A Thing Of The Past For Google Link
User experience is central to most online retailers. A slow website with bloated code, an overwrought UI and plenty of popups is everyone’s nightmare, especially on a mobile device.
The “mobile-friendly” label was introduced by Google in late 2014 as an attempt to encourage websites to ensure a good mobile user experience. After widespread adoption of responsive design, the mobile-friendly label is being retired by Google in favor of the AMP label.
11 This is how AMP results show up in Google mobile currently. (Image: Myriam Jessier) (View large version12)
AMP pages could be featured in a carousel and are labelled with a dedicated icon, highlighting them in search results. The search giant has recently stated that AMP would take precedence over other mobile-friendly alternatives such as in-app indexing. However, AMP is still not a ranking signal13, according to Google Webmaster Trends analyst John Mueller.
AMP: Because Mobile-Friendly Doesn’t Cut It Anymore Link
Media queries adapt the presentation of content to the device. However, the content of the page itself isn’t affected. In contrast, AMP helps make mobile web pages truly fast to load, but at a cost. Developers, designers and marketers will have to learn how to create beautiful web pages that convert using a subset of HTML with a few extensions.
The premise of AMP14 is that mobile-optimized content should load instantly anywhere. It’s a very accessible framework for creating fast-loading mobile web pages. However, compatibility with the AMP format is not guaranteed for all types of websites. This is one of the realities of a constantly evolving project such as AMP. The good news is that many of the arguments against AMP for online retailers no longer hold up.
AMP pages are now able to handle e-commerce analytics thanks to the amp-analytics variable. With this variable, statistics are available to analyze an AMP page’s performance in terms of traffic, revenue generated, clickthrough rate and bounce rate. According to the AMP project’s public roadmap15, better mobile payments are planned, after the addition of login-based access, slated for the fourth quarter of 2016.
Product and listing pages are supported in AMP, and they show great potential to add real value to the online customer journey. Keep in mind that 40% of users will abandon a website if it takes longer than 3 seconds to load16. Worse yet, 75% of consumers would rather visit a competitor website than deal with a slow-loading page.
Some of the drawbacks that have been noted are mostly due to the fact that AMP for e-commerce is rather new. There are a few concerns about the quality of the user experience offered by AMP e-commerce pages because some e-commerce functionality is not yet available, such as search bars, faceted search filters, login and cart features. However, frequent updates to the AMP format are planned, so this shouldn’t be a deterrent to those looking to implement it.
There has been some grumbling about the format among marketers. AMP relies on simplified JavaScript and CSS. As a consequence, tracking and advertising on AMP pages is less sophisticated than on traditional HTML pages. That being said, the main drawback is that implementing AMP pages effectively will take time and effort. The code is proprietary, heavily restricts JavaScript (iframes are not allowed, for example) and even limits CSS (with some properties being outright banned).
How to Develop AMP Pages for an E-Commerce Website Link
To ensure that your website is AMP-compliant20, check the instructions provided in the AMP project’s documentation21. Keep in mind that AMP pages should be responsive22 or mobile-friendly. A best practice would be to test the implementation of AMP pages against your current mobile website using a designated subset of pages. This will give you a sample to determine whether AMP adds value to your business.
You don’t have to make your entire website AMP-compliant. Start upgrading the website progressively: Pick simple static-content pages first, such as product pages, and then move on to other types of content. This way, you can target highly visible pages in SEO results, which will lead to a big payoff for the website without your having to deal with pages that require advanced functionality not yet supported by AMP.
If your website uses a popular CMS, then becoming AMP-compliant could be as easy as installing a plugin.
Magento
The AMP extension by Plum Rocket23 automatically generates AMP versions of your home page, category pages, product pages and blog pages. An interesting feature is that the AMP home page isn’t just “converted”; you can edit it in Magento’s back end.
WordPress
AMP for WP24 is a plugin that lets you create custom AMP designs without having to code. You can customize the logo, header, footer, images and more. It is compatible with WooCommerce and AdSense. The plugin generates AMP versions of your home page, blog articles, WooCommerce shop, products and categories pages.
Shopify
Nothing yet, but it’s under way!
A Step-By-Step Guide To Implementing AMP On Your E-Commerce Website Link
Let’s break down the process according to the customer journey. AMP offers a selection of prebuilt components to help you craft an enjoyable user experience on an e-commerce website (along with some evolving tools to help you collect data in order to improve it). You can implement four major AMP elements along key points in the customer’s purchasing journey, including on the home page, browsing pages, landing pages, product pages and related product widgets:
product descriptions,
reviews,
product shots,
navigation.
The entire purchasing flow can’t be 100% AMP-compliant yet, so you’ll have to plan a gateway to a regular non-AMP page for ordering and completing purchases.
Users will often start their purchasing journey on a website’s home page or a product category page, because these pages are prominent in search engine results. These pages are great candidates for AMP, as eBay has shown25 by making many of its category pages AMP-compliant. Typically, category pages are static and showcase products for sale. The amp-carousel feature26 offers a way to browse other products in a mobile-optimized way. These products can be organized into subcategories that fit the user’s needs. You can view the annotated code to create a product page over on AMP by Example27.
28 AMP e-commerce home page (Image: AMP by Example)
After browsing to a category page, the next step for our user would be to find an interesting product and click on it. In an AMP-compliant flow, this would lead the user to an AMP product page29.
Showing related products36 benefits the retailer’s bottom line and the user’s experience. The first product that a user browses to isn’t always the one that fits their need. You can show related products in AMP in two ways:
Statically publish a list of related products.
Generate the list on the fly using amp-list37 to fire a CORS request to a JSON endpoint that supplies the list of related products. These related products can be populated in an amp-mustache4138 template on the client. This approach is personalized because the content is dynamically generated server-side for each request.
Personalization is a big deal in e-commerce because it increases conversions. To dip into personalization in the AMP format, you can leverage the amp-access39 component to display different blocks of content according to the user’s status. To make it all work, you have to follow the same method as we did with the amp-list40 component: Fire a request at a JSON endpoint, and then present the data in an amp-mustache4138 template. Keep in mind that personalization doesn’t have a leg to stand on without reliable data. Google has been actively extending the tracking options available in AMP.
Sidenote: In case you see cdn.ampproject.org in your Google Analytics data, this is normal for AMP pages; cdn.ampproject.org is a cache that belongs to Google. No need to worry about this strange newcomer to your Google Analytics data!
AMP now supports some analytics products, such as Adobe’s and Google’s own. The type attribute will quickly configure the respective product within the code. Here’s an example of type being used for Google Analytics:
<amp-analytics type="googlenalytics">
And here are the types for some of the most common analytics vendors:
Adobe: adobeanalytics
Google Analytics: googleanalytics
Segment: segment
Webtrekk: webtrekk
Yandex Metrica: metrika
Google Tag Manager44 has taken AMP support one step further with AMP containers. You can now create a container for your AMP pages.
45 Google Tag Manager’s AMP container (Image: Myriam Jessier)
More than 20 tag types are available out of the box, including third-party vendor tags. Alongside a wider selection of tags, Google has provided built-in variables dedicated to AMP tracking, making it easier for marketers and developers to tag their pages.
46 AMP tracking options in Google Tag Manager (Image: Myriam Jessier)
If you are not using Google Tag Manager, you can implement your tag management service in one of two ways:
endpoint
This acts as an additional endpoint for amp-analytics and conducts marketing management in the back end.
config
This manages tags via a dynamically generated JSON config file, unique to each publisher.
The endpoint approach is the same as the standard approach. The config approach consists of creating a unique configuration for amp-analytics that is specific to each publisher and that includes all of the publisher’s compatible analytics packages. A publisher would configure using a syntax like this:
Many online retailers rely on advertising or showing related products throughout their website to boost revenue. The AMP format is bootstrapped to show ads through <amp-ad> and <amp-embed>. The documentation is quite clear47 on how to implement ads, and the good news is that a wide variety of networks are already supported. Although iframes are not allowed in AMP, two embed types support ads with <amp-embed>: Taboola and Zergnet. If you plan on using ads in AMP, follow these principles48 in your development work:
The previous step was a tricky one because it entails maintaining a seamless user experience while the user transitions to a full HTML page. The process should be fast and consistent for the user. An experience that isn’t consistent with the preceding AMP journey could hurt conversions. If your website is a progressive web app, then amp-install-serviceworker49 is an ideal way to bridge both types of pages within the customer journey, because it allows your AMP page to install a service worker on your domain, regardless of where the user is viewing the AMP page. This means that caching content from your progressive web app can be done preemptively to ensure that the transition is smooth for the customer, because all of the content needed is cached in advance. An easy way to experience the entire AMP e-commerce experience is to head on over to eBay50; see how the company handles the transition from AMP to an HTML checkout process.
AMP works within a smart caching model that enables platforms that refer traffic to AMP pages to use caching and prerendering in order to load web pages super-fast. Be aware of this when analyzing traffic and engagement because you might see less traffic to your own origin when AMP pages are originally hosted (this is why we referred to cdn.ampproject.org in Google Analytics data). The balance of traffic will most likely show up through proxied versions of your pages served by AMP caches.
AMP encompasses a lot of best practices for building mobile web pages. Incorporate mobile best practices as part of your regular development lifecycle.
Less forking in code
If you follow AMP best practices when building regular pages as well, you can reuse most of the UI components between AMP and non-AMP pages. That means less forking (except for the JavaScript-based components).
Adding AMP’s ecosystem to one’s internal search would be a very interesting prospect for many online retailers.
Mind you, there are some complex parts:
Infrastructure components
Things such as global headers and footers and tracking modules have some JavaScript, which is a no-go for AMP. This adds complexity to development but can be worked around.
Tracking
AMP provides user-activity tracking through its amp-analytics component6242. The component can be configured in various ways, but it is still not sufficient for the granular tracking needs of most online retailers.
However, once you get past the internal hurdles, the payoff can be great. Check out the examples provided by eBay for camera drones63 and the Sony PlayStation64. (Use a mobile device, of course, otherwise you will be redirected to the desktop version.)
SEO experts are pushing for AMP adoption because some see it as a mobile-visibility asset to be leveraged. Here are some SEO points to ensure you get the most out of AMP:
Host AMP pages on the same domain as other page versions.
The Google AMP cache is a proxy-based content delivery network for delivering all valid AMP documents. It fetches AMP HTML pages, caches them and improves page performance automatically.
An AMP page is served to the user from the Google AMP cache, and it will have a different URL so that duplicate content issues are avoided. If you have both AMP and non-AMP versions of your pages, use the <link rel="canonical" href="[canonical URL]" /> tag on the AMP page and <link rel="amphtml" href="[AMP URL]" /> on the regular page. For a standalone AMP page (one that doesn’t have a non-AMP version), specify it as the canonical version: <link rel="canonical" href="https://www.example.com/url/to/amp-document.html" />.
One of the most common URL structures is to add /amp/ to the path of the URL.
An e-commerce website can’t be 100% compliant with AMP, but there are benefits to adopting the format early on. Online retailers looking for an edge against fierce competition might be wise to turn to this format to grab the attention of mobile customers and nudge open their wallets. More and more websites are converting to the AMP format to increase or maintain their mobile traffic. For an online retailer that has a multi-channel or mobile-first strategy to acquire and retain customers, AMP might be a great way to future-proof their online marketing efforts.
The world is constantly evolving with frameworks, such as the Internet of Things (IoT) and virtual reality (VR). These and many others are opening opportunities to rethink how we approach prototyping: They introduce avenues to marry the digital software with the tangible aspect of the overall user engagement.
This two-article series will introduce readers of different backgrounds to prototyping IoT experiences with minimum code knowledge, starting with affordable proof of concept platforms, before moving to costly commercial offerings.
“Part 1: Building the Hardware” will identify the problem, the criteria for selecting hardware and, finally, how to put the different pieces of equipment together.
In “Part 2: Configuring the Software,” we will continue the discussion by writing the code to control the hardware and connect the hardware to the Internet, and we will design custom interfaces to display the collected data.
We will do this by going over a personal experience I had as a user experience designer while learning the basics of an IoT platform named “Adafruit IO”. This will be a nice introductory case study.
The following are some assumptions about you:
You can read code and may have even written some on your own (we’re not going to learn coding basics here).
You have some understanding of circuitry and electronics. For more on this, see the “Resources” section.
You are curious and like to explore and tinker.
Disclaimer:I am not an electronics engineer or a developer. Please always be careful when exploring electricity and hardware. This tutorial is meant to inspire you to do additional research before finding what works for your circumstances!
IoT talk is sometimes unnecessary complex. To reduce the jargon, I will use some reader-friendly terms, as defined below.
board
You can think of the board as a mini-computer that holds the software in code form (often called firmware) that is responsible for determining how different sensors and the board itself behave in response to inputs from and outputs to their environment. A more technical term is microcontroller unit (MCU).
cloud
There are a ton of definitions and as many philosophical debates regarding this term. For the purpose of this series, let’s define it as any remote servers, plus inbound and outbound connections to and from them.
ecosystem
This is a collection of hardware and software that create a unique overall experience for the end user.
rig
The board, power supply unit and any attached sensors form a cohesive hardware unit.
On a cold winter day, I read an article on smart homes being the future, which immediately inspired me to turn my home into a smart one. This translated into several commercial product purchases, including devices from the Nest family, which just whet my appetite.
Controlling my air conditioning and furnace and detecting possible carbon monoxide emissions were not enough! I wanted to go further by having monitoring capabilities over my home security. This includes:
tracking doors opening and closing,
verifying the occurrence of a fire or flood,
looking at temperature and humidity,
capturing movement in my garage.
Getting to the point of picking Adafruit IO as the solution was not a simple journey. Before deciding on that platform and the HUZZAH ESP8266 board, I tried several other solutions, with varying success:
I gave this a go before even having selected the cloud platform for it. My experience showed me that programming the board requires additional setup, and pushing code to it is often unreliable. For the rigs that I got up and running, I noticed the Wi-Fi connection dropping completely after only a few days.
This is a beautifully designed package of sensors and central wireless hub. It is part of an ecosystem that comes with a handy mobile app and support for third-party products7. I spent considerable time trying to get the sensor-to-hub communication to work, but unfortunately, even with the help of the support team, it did not work. I attributed this to my home being a black hole for all things wireless.
This ecosystem offers dedicated development tools, various IoT-based boards and robust integration with libraries. I gave it a try and was impressed with the performance and flexibility for the end user. I hit roadblocks when integrating Blynk9, a platform for controlling devices via drag-and-drop interfaces. Though I am putting this approach on hold, I look forward to exploring the Particle-Blynk combination again soon.
My vision was to have multiple sensors that could be viewed and controlled from a computer or mobile device independently at any time. To accomplish this, I needed both a Wi-Fi-enabled hardware board and a software platform that could talk to it and any attached sensors.
I decided to be more strategic in my choices, so I came up with a list of criteria, in order of priority:
low cost
For scalability, I wanted to keep the rig (i.e. board, sensors and power supply) in the $25 to $30 range — 50% less costly than commercial offerings.
small board
I wanted a small size for easy mounting and to have enough space to fit multiple sensors in a custom enclosure.
support community
In case of problems, access to learning resources and a community are key.
wired power
In tests, a 9-volt battery lasted only a day. Solar was out of the question due to poor sun coverage of the home. Thus, the board had to be able to be wired to an outlet.
low learning curve
Though I have some development experience and hardware hacking knowledge, I didn’t want this to be an arduous project.
After exploring the three approaches mentioned further above, I ruled out the following additional equipment, based on the five criteria. Keep in mind that I am giving you the high-level details — a whole article could be written on selecting a board!
Offers both wired and wireless Internet connectivity, expandable RAM and onboard memory. The board has a Linux-based distribution, making it a powerful networked computer.
The price of the controller ($69) and the bulky size proved to be too limiting. Also, I didn’t need something so powerful. I ended up buying one to test out for a garden watering project.
In addition to offering wired and wireless connectivity, it has HDMI and audio ports, Bluetooth integration and support for use of a custom-sized SD card with different operating systems.
While I could load a Linux-based operating system and use the Python language to accomplish anything, that’s not what I needed. I ended up using this platform for other projects. The $40 price tag and the bulky size were also limiting factors.
This $5 board packs a big punch. The powerful CPU and large RAM made it a strong contender, as did the small size and large number of GPIO pins.
Two things nixed this board. It doesn’t have on-board Wi-Fi, and so requires additional equipment. And because it is very popular, finding this board is hard. In the US, it is sold only at Micro Center13, which limits it to one per home per month. (Note: At the time of writing v1.3 of the board with on board Wi-Fi and Bluetooth was not yet available.)
Side note: For more information on choosing a board for your hardware prototyping project, you can consult the excellent “Makers’ Guide to Boards14.”
Further researching led me to Adafruit’s HUZZAH ESP826615 board, which is but one variation of the ESP8266 chipset; there are others, such as the NodeMCU LUA16. Each has unique capabilities, so choose wisely. Here is why I selected the HUZZAH:
the price ($10, not counting shipping);
12+ digital input and output pins and 1 analog pin;
an on-board reset button
voltage shifting from 3.3 and 5 volts (but it’s primarily a 3.3-volt board)
a dedicated IoT service (Adafruit IO) with UI building blocks.
Deciding to start small, I wanted to build a sensor that tracks whether a door is open. The rest of this first article will focus on the hardware for this use case, but much of the wiring will scale to other types of sensors.
It’s best to get one that supports 3.3 and 5 volts, in case you want to explore rigs using the original ESP8266 ESP-01 board in the future. It must offer TX and RX LEDs for easier troubleshooting while pushing code to the board. This is used to push the code from the programming computer to the board.
Before getting to the details of how to put the rig together, let’s talk about what the goal is. By the end of this first article, you should have something similar to what you see below. With this setup, you will have a mini-computer (the board) capable of collecting sensor data from your environment and communicating it to the cloud (Adafruit IO) over Wi-Fi.
You can attach multiple sensors, but be aware of the power draw (i.e. how many milliamps each sensor uses) and the length of wire needed to connect them. (View large version3229)
The first step is to assemble the HUZZAH board by soldering on its pin headers, including both the board leg headers and the FTDI header. Adafruit has a step-by-step tutorial30 on this.
When you are soldering the first leg header, ensure that the board is not tilting one way, which would result in the pins being soldered at an angle. A trick I used is to put a bit of putty under the board to even it out as it is being plugged into the breadboard.
Once you have soldered all of the headers, insert the board in the breadboard with the antenna (the wavy gold line) facing outwards.
Insert the supply at the opposite end of the breadboard, with the top and bottom legs fitting in the + and – breadboard rails. This is how power will be passed to the breadboard.
Next, set the yellow jumpers for both rails to 3.3 volts, which is the voltage used by the HUZZAH board.
Note: Depending on your breadboard, the – and + might not match the alignment of the power supply jumpers. That’s fine as long as you remember that the power supply dictates which breadboard rail carries which electrical signal!
Connect the + to the + rail on the breadboard (the power).
Connect the – to the – rail on the breadboard (the ground).
Connect the S to the #5 on the board (digital input pin).
A 9-volt battery is used here to illustrate a power source, but most such batteries might not have adequate power for your rig because they typically carry 400 to 600 milliamps. (View large version31) The power adapter is plugged in here but not powered on. Use different colored wires to distinguish between connections. (View large version3229) Notice how each pin of the sensor is labeled. (View large version33)
As a last step, plug the 9-volt adapter into a power outlet, then into the breadboard power supply. Push the white button. If everything is correctly wired, you should see several lights flicker on, including for the power supply (the green one), the board power (red), the Wi-Fi (blue) and the sensor (red if the magnet is touching the sensor).
At this point, you can start writing the code for the rig, but I find this is a good opportunity to test the mounting of the container box. This is not a permanent mounting, but a trial run to gauge the rig’s overall dimensions and the best fit. Before doing that, you need to take some steps first.
Step 1: Using your soldering iron, melt one hole in the left side of the container for the power adapter plug, and three smaller ones on the right for the individual sensor wires.
Warning: Make sure to do this in a well-ventilated area, so that you don’t breathe in fumes. After that, clean your soldering iron’s tip with a nonabrasive sponge.
A similar box can be bought on the cheap in any large store. (View large version35) Carefully size the holes based on your wires’ thickness. (View large version36)
Step 2: Put the entire rig in the container, and pass the cables through the holes.
While this looks well insulated, it is not suited for outdoor use! (View large version37)
Step 3: Close the container, and mount it on the wall with the putty. Tapes of various types won’t work well. Alternatively, you could punch holes in the bottom of the container to mount with screws, but make sure they are insulated with electrical tape to avoid short-circuiting any electronics.
I have also tried using hot glue. I think it is messy, but it is not all that more expensive, and you can pick one up on the cheap38 if you prefer that method.
Put putty only on the bottom of the container and the sensor. (View large version39) You can stack multiple magnets on the door to match the proper distance from the sensor. Make sure it does not touch the glass tube. (View large version40)
Step 4: Use a combination of LEGO pieces and putty to mount the sensor and the accompanying magnet to the door.
For a firmer fix, you can put a LEGO piece on top of the two blue blocks. (View large version41)
Now that the rig is all wired up, you can connect to it with the FTDI cable and start adding the code that will make the sensor work.
Using a 5-volt 1-amp power adapter (rather than 9 volts and 1 amp) won’t cut it. In my tests, the rig never powered on this way. I attribute this to the voltage conversion in the breadboard’s power supply.
In trying different hall sensors, I was getting readings all over the board. See if they work for you, but my preference remains a reed switch because of its large surface area for interacting with the magnet.
In this first article of our two-part series, we’ve identified the problem (home security), assessed the merit of an IoT setup, and discussed the rationale involved in selecting a particular board. This was followed by a step-by-step guide on how to put together all of the hardware components into a working rig.
In doing so, we’ve learned the basics of electronics. In the second and final article in this series, we will add code to the rig we’ve built here, so that we can start interacting with the environment. Then, we will build custom user interfaces to view the data from anywhere, while discussing at a high level the security implications of the software configuration.
21
