Experience Design for Machine Learning and AI

Just over thirty years ago, the first Terminator movie hit the theaters. It was a game changing moving starting starring Arnold Schwarzenegger as the Terminator, a cyborg assassin sent back in time from 2029 to 1984 to kill Sarah Connor, whose son will one day become a savior against machines in a post-apocalyptic future. An artificial intelligence network will become self-aware and initiate a nuclear holocaust.

The storyline seemed far-fetched back in 1984 but today seems surprisingly accurate. With only 12 years left until the singularity event portrayed in the movie, we are far down the path to smarter machines. Elon Musk has been famously warning of an impending disaster. Machines already with access to all the world’s information are starting to learn, change and adjust themselves to the world. While I hope we will not see a future at war with the machines we, as a society, need to start engaging with this new reality.

Recently, the concepts arounds machine learning and artificial intelligence have been elevated to forefront of tech world. Every service we use is getting smarter and smarter.  Each time you search for a concert or dogfood, machine learning algorithms are deployed to improve the quality of your results. They are becoming smarter with each question. In addition to text search, the quality of voice and visual search tools are improving exponentially.

I recently took part in a hackathon where teams were tasked with creating a new service with the help of Machine learning. With no experience in machine learning it seemed a daunting task. Our team made up of three developers, a data scientist and myself representing design and product had never used machine learning tools.  Time boxing work can be a magical thing and within 72 hours our team took a concept from design to a working prototype.

Our solution was built to solve a challenge of the retail world. Shoppers like search by color and style but brands generally organize products individually by category. Using visual recognition, we built an app that allow retail customers to capture images of head-to-toe looks from social media or their phone. The app then used visual recognition to find styles and colors from our brand that would be a good match. Using images and tags from target products we were able to train our app producing matching results with surprising accuracy.  I was floored when hours before our deadline we ran through the demo journey end to end and it worked!

With new tools such as IBM Cloud and AWS Tensorflow, developers can quickly implement advanced machine learning solutions without the complex math used by data scientists.  These tools have changed the game for developing intelligence in user experiences. Organizations can quickly deploy these algorithms without needing to understand the underling intelligence layer.

The experience was a bit shocking to me. As someone who with a career building systems intended to improve lives, it is alarming to imagine how easily we can be surrounded by smart machines without a deeper understanding of how they make decisions and how we should manage them.

Designing for the Machine

As user experience designers, how do we arm ourselves for a future with smart machines? As far back in 1959, Arthur Samuel described machine learning as: “the field of study that gives computers the ability to learn without being explicitly programmed.” Adoption of the technology has slowly grown over the year but has recently exploded into every industry. While ML and AI is being applied across every platform, design for human interaction with these systems has a lot of room to catch up.

https://www.nvidia.com/en-us/deep-learning-ai/industries/

Call to Action

With such low barrier to entry to use, designers need to take an active role in shaping the experience powered by the machine learning rather than letting the tools define the experience.

The two main categories of machine learning include supervised and unsupervised. Supervised learning requires examples of correct solutions to train the algorithm. For example, when you sort your Gmail and tag some into the spam folder, machine learning algorithms are improved by adding to the data set.

Unsupervised learning, by contrast, uses algorithms to categorized data without knowledge of expected results. Sets of data can be grouped. This can be helpful in analyzing data and identifying cohorts needing further study. For example, groups of consumers can be categorized into cohorts based on their past behavior without intentionally knowing what attributes are important.

Design Best Practices

Challenges arise when you need to get users to take action to improve an algorithm. In the example of an email spam filter, the user, not knowing about the benefits of sorting their mail, deletes the email and loses the chance to have less spam in the future. As a user, the quickest route might seem most expedient but small changes in your interaction might improve your life in the long run. Designers need to consider how to craft messaging and flows to correctly position learning features so that they appear to be in service of the user, not the machine.

The Google Nest has been a leader in the smart home segment. With learning algorithms, Nest promises to reduce costs and improve our comfort. It is a great example of an elegant, streamlined onboarding process. One thing the designers at Nest have done well is explain how interacting with the thermostat will make it smarter. This little incentive to the home owner to make more thoughtful decisions will create better results down the line. More recently, the Nest app has implemented a geo-fencing feature that allows it to set Home/Eco mode based on your location. On iOS users must grant location access to each app which many hesitate to allow. Nest could improve their messaging to explain the benefits of geo fencing and how it will make the Nest smarter and save you money.
As experience designers, we need to develop best practices to ensure we apply machine learning ethically.  It is important to communicate with the end user how their interactions will used. With a goal of transparency, we must communicate immediate benefits to the user, longer term benefits to the community and the impacts of data storage over time.

 

Advocacy Marketing — How to create lasting connections with consumers

The marketplace continues to evolve at an accelerating rate. It is easier to create a message and deliver it to consumers but harder to cut through the clutter. Today, two-thirds (67%) of Americans report that they get at least some of their news on social media — with two-in-ten doing so often. In addition, on each social medial platform, paid content is increasing as brands increase spending to get their message in front of consumers. Confidence in social is eroding as the ratio of trustworthy information diminishes and stories of fake news proliferate.

What is a brand to do in these times? Many are turning to advocacy programs to share their messages. Advocacy Marketing focuses on developing relationships with consumer advocates who are willing to share their views about the brand. More than 80% of shoppers research online before buying, and having people publicly advocating for the product gives these researchers something to find and study.

Why do you think a referral or advocacy program is important?

The right advocacy programs can be a game changer in connecting the consumers to brands. It is increasingly difficult to differentiate products. Brands need to break through traditional marketing channels and connect at a deeper level. Almost all purchases are now are informed by online research. Before making a purchase, consumers are looking for a recommendation from a friend or influencer. An advocacy program can be a powerful tool in supporting and growing the organic referral process. Potentially more important, programs can increase the engagement with existing customers to reducing churn and acquisition costs.

What is in store for the future of referral or advocacy marketing?

Driven by technology, changes in marketplace continue to accelerate. Digital will become even more omnipresent at all touchpoints. The ease of comparing products with consumer reviews will increase as more tools are developed to aggregative information.

Brands will become better at building holistic consumer relationships at every touchpoint. For advocacy program, the differentiator will be utility. First and foremost a program must deliver immediately value. It must reduce a paint point or address a need. Once the program becomes a regular tool in the consumer’s toolbox it can be leverage to create true emotional response. Emotional connections will then be leveraged to create an enduring attachment can create huge increases in lifetime value.

Increased personalizing will be a common theme as brands become more sophisticated. Everyone wants to feel special and be rewarded. Brands will respond with more tailored products. For example, customers with heathy lifestyles will expect that they should be recognized and rewarded. Using technology that is seamless and invisible consumers will take advantage of the programs that make their lives better with little investment.

How to measure your referral or advocacy program?

Developing KPIs for each stage is critical to managing a successful program. In the startup stage, consumer engagement is the primary driver. Acquiring new users and keeping them engaged through whatever means necessary with be the initial focus. As the product matures, developing an emotional connection will become more of the focus. Using consumer interview and survey (NPS) can help measure the success of new features. Referral count is the long-term measure of success that completes the loop. Since good consumer advocates have an outsized influence over brand perception, it is critical to continually monitor sentiment and take action if there is any significant negative trends. Brands can deploy powerful social media monitoring tools (Sprout Social, Spriklr) to track trends in real time and take action if needed.

Design for Physical Spaces and New Interaction Models

 

The Internet–Of–Everything

The first internet revolution was focused connecting to the world through the window of your laptop or smartphone. Today, the internet has become woven throughout our physical world. No longer is the internet confined to web pages. Today we control our lights and locks with a voice command. Ordering a pizza is only slightly harder than thinking about it. As experience designers, we need to consider each thread of the digital fabric of user experience.

Interaction Design in this Brave New World of Sensors and Environments and Immersive Experiences

What is an immersive experience? It is a technology, imagery or physical space that actively engages one’s senses and creates an altered mental state. Traditionally theaters broadcast stories to audiences. With immersive experiences, the theater goer becomes part of the cast and touch, feel, speak to the story.

At their core, all compelling experiences have a strong storyline. Interactive theater, museum installations and great retail all engage people with a good story and evoke and emotional response. That storytelling and emotion can be used to deeper connection and provoke action.

The environmental canvas includes everything that interacts with the five senses. Digital elements include displays, lighting and sensors combined to communicate, listen and learn. Perhaps most exciting, people can be empowered by technology to support a deep, personal interaction.

The retail world has long measured store traffic flows to optimize product placement. Now with beacons, WiFi tracking and RFID, companies track physical shoppers almost as closely as traffic on a website. Similar to optimizing web conversion funnels, we can now optimize customers’ every step through a store or brand space. When a shopper enters a store what emotions should they feel? Signage, displays and in-store associates are all additive to the experience. Too much information can be overwhelming, and put off shoppers. To little engagement and the shopper can feel like they were ignored. Each immersive experience should be created with a plan to measure and adjust similarly to new features on a web experience.

Retail Earthquake

The retail has undergone massive seismic shifts. Today, more than 75% of consumers are referencing their phones before making an in-store purchase. To evolve their business model, many brands are working to support showrooming and webrooming. As retailers and consumers continue to evolve their behavior, winning brand experiences will frictionlessly transition from digital to physical and back focused on developing the relationship with every interaction.

With the rise of Amazon, brands are moving to set themselves apart from competition with immersive experiences. Leading brands like Nike, Target and Tesla are rethinking retail. The new retail experience is less transactional and more emotive. Explaining product benefits and demonstrating how it fits into your life elevates the value of the brand and creates a lasting bond.

This trend will continue to evolve and change expectations. Associates will no longer be solely focused on driving sales but building authentic relationships. Today experience designers need to consider the entire journey and empathize with the small pain points that contribute to everyday decision making.

What Draws Brands and Marketers to Immersive Experience?

Today’s consumers are distracted by a myriad of messages competing for their attention. Digital displays, push messaging and advertising all compete for attention across each media channel keeps consumers from engaging deeply with any message. To truly build brand affinity brands must evoke an emotional response. An immersive experience can create this focused space.

Beyond dedicated space, emerging technology should be considered to create a similar reponse.

Example

• Urban Environment: street lights, parking meters and smart bicycles
• Intimate / Personal: Smart watches, Smart wear clothing

An immersive experience endeavors to, for a moment in time, push away those distractions and fully engage the audience/actor. Once the stage is set, storytelling become the element that connects. You have to create a narrative that feels user-driven from every angle while controlling the space from behind the curtains.

This can be challenging when the audience is allowed to explore on their own terms. Rather than try to create a complicated backstory for the audience—which usually leads to boring exposition—give them something to do.

 

 

 

Case Studies

Google Store

One of the biggest hurdles to engagement is to compel the audience to take a action and start to engage with an experience. How do you convince participants to take that first step? In Google’s pop-up shop in Soho, they designed an urban kitchen and living room set complete with kitschy candles and Google brand cereal. A product expert stationed in each set engaged the audience and took Google Home through its paces

 

 

 

 

 

 

 

 

Sonos

A few blocks away from the Google pop-up the Sonos Flagship store provides an excellent example of a self-guided immersion. Experience pods occupy the center of the store representing different room types such as living room, study and kitchen. When the visitor enters the pod, a tablet has a clear play button to start the experience. Audio and visual cues then walk the visitor through various use cases. The experience feels fun and effortless. It creates a vision of a better life that is attainable if you just buy a Sonos system. The experience includes a clear call-to-action, full 360 degree immersive environment and a compelling storyline.

Clearly the world is moving towards experience over transactional interactions. Crafting an seamless interaction is more challenging yet more rewarding ever.

Questions? Comments. Please let me know