Overall Lack of Engineering Engagment

Initial talk around the watercooler highlighted a lack of overall engagement from the engineers. The engineers were spread across the US and Europe, so there was a cultural, time zone, and language barrier that kept the teams from feeling connected. Because of this, the engineers didn’t speak up in meetings, they didn’t ask a lot of critical questions, and they generally did the bare minimum to pass acceptance criteria. No one seemed excited to work on SmartShop.

Lack of Process Knowledge

Because of the lack of engagement, engineers had little insight into why they were building what they were building. The engineers were constantly pushing back on the UX team because they didn’t feel certain features were necessary. There was a general lack of understanding of what users were doing and how they were interacting with the software, leading to conflicts between engineering and the UX team.

Lack of Design System Knowledge

Engineers had access to a design system. In theory, this would make them fast. Unfortunately, no one seemed to be using the design system. No one was using the components, no one was using the variables for colors, spacing, or typography. The result was inconsistent implementation leading to inconsistent interfaces for users.

Nine pods worth of engineers is a lot of people. All told, that was about 70 engineers. To make matters worse, the teams were spread across countries with different time zones, languages and cultures.

We needed to align the teams so that:

• Everyone felt like they were working towards a common goal.
• There was a consistent understanding of the business processes.
• There was a consistent understanding of the users and how they wanted to use the tool.
• There was an improved knowledge of the design system and how to use it.
• Everyone felt excited to work on SmartShop.

There was clearly a lot that wasn’t working. Where should we start? What action items would have the most impact? To better understand just how bad each of these issues were, the UX team decided to create a quarterly survey of 10 simple questions for the engineering teams.

Not only would this show us what areas needed the most improvement, but we would be able to track our progress over time and see how our actions were

What Do We Want to Know?

As a UX team, we wanted to look inward. Were we to blame? Were we not doing enough to help the engineering teams understand the larger picture? We wanted to look inward and try to hit a few key topics:

We wanted to make this simple, quick, and easy without causing disruptions for the engineering teams. Since this is something we wanted to do every quarter it needed to be dead simple.

• Number of questions capped at 10.
• Scale of 1-6, so answers swing in either direction.
• Free-text fields enable detailed feedback on improvement suggestions.
• Follow-up every quarter.

The Engineering Survey

• The wireframes provide enough information to accurately implement user stories (1-6 scale)
• Using the design system out of the box takes care of implementation/styling issues (1-6 scale)
• I’m satisfied with the availability of the UX team to answer questions in a timely manner (yes/no)
• I’m satisfied with the overall UX process as it applies to implementing user stories (1-6 scale)
• I am satisfied with the level of knowledge I have on the end users (yes/no)
• How much interaction with UX would you like to see during a sprint? (multiple choice)
• In your opinion what about the current UX process makes your job difficult (free text)
• In your opinion, what about the current UX process makes your job easy? (free text)

The initial results were enlightening. Generally engineers found working with the UX team to be good. Our wireframes were solid and we were generally available when needed.

But it was clear we had a lot of work to do.

• 84% felt they had little insight into end users.
• 42% found the design system challenging to use.
• 26% felt we should improve our processes
• Many wanted the UX team to be more present.
• Many felt we were slow to respond when changes needed to be made.

UX Process Improvements

Clearly, 26% of SmartShop engineers thinking we can improve our UX process is a significant number. After reviewing results and following up with different engineers we decided to:

• Break our team of the 5 designers into dedicated UX resources, by assigning each designer to two specific pods.
• Make a point to attend as many SCRUM ceremonies as possible, including daily standup, planning, and retros.
• Use planning sessions to review wireframes and answer any questions.
• Created UX acceptance Criteria for user stories.
• Set up UX/QA touchpoints to set propper expectations across the teams.

Design System Improvements

Prior to this survey our design system had some challenges. It was a shared repo that was initally intended to be used by many teams as a shared opensource, contributiion model. Unfortunately many teams at GE had copied this repo and used their own flavor within their apps.

• At this point, SmartShop was one of the only teams still using the shared repo. We figured we would cut our losses and give up on the open source model and pulled the design system into our own app.
• Not everyone was a fan of He-Man. Talking with various teams, we found that they thought “Skeletor” was silly. We re-branded Skeletor to SmartShop UI.
• We worked closely with the lead engineers to implement a component library using Storybook, owned exclusively by the SmartShop team.

Engineer Engagagement Improvements

84% of develoeprs felt they didn’t understand the users. This was the biggest challenges we faced. How do we get 70 people spread across two countries to feel like they truly understand the users?

• We implemented a “ShopTalk” series at each of the sits where the UX team spent about an hour each month reviewing various business processes, answered general questions, and asked engineers to come up with topics they felt they didn’t understand for future sessions.
• We created a “Mini Brief” that helped to explain different development features, the business impact, and defined the applicable personas and their pain points related to this feature. We reviewed these in all sprint planning meetings and attached each to the corresponding user stories.
  
• We invited functional teams to the sites to explain processes and provide insights on the users. These teams would bring actual components and other things fromt he shop so the larger engineering team could get first hand experience handling parts.
• We made a rule that each research trip going forward would have at least one engineer in attendance to help build empathy for users and undestand their pain points and needs.
• During each research trip we would document our experience with a fun video blog of what we learned that day and share with the team back home.

The SmartShop UX team ran this survey a total of four times, Q4 2018 | Q1 2019 | Q2 2019 | Q4 2019. Over that time we saw interesting results.

You might notice a survey and data missing, along with a dip in some of the statistics toward the end of 2019. At that time, the team was going through dramatic changes, being affected by the 737 Max groundings. Funding was decreased and the engineering teams were shuffled around, leading to turnover and other challenges. These external issues along with COVID-19 ultimately led to the defunding of Smartshop in early 2020.

The wireframes provide enough information to accurately implement user stories.

After the first survey the UX team started delivering more red-line spes to review during handoff as well as set up time to review wireframes and prototypes as we were designing so engineers knew what to expect later.

Using the design system out of the box takes care of implementation/styling issues.

The biggest roadblock to fully embracing the design system was the engineers didn’t feel like they had ownership over it. After the first survey we pulled the design system and all components into the main SmartShop repo that our development team owned 100%.

Once doing this, engineers were able to modify components as needed, contribute back to the system, and use the components without fear of outside teams introducing breaking changes.

In addition to this, we implemented Storybook to build an interactive component library with detailed documentation for the engineers to understand how each component worked and to see and modify properties on the fly.

An added benefit of using Storybook was that the UX team could see production ready code and offer changes to styling before it ended up in production.

I’m satisfied with the availability of the UX team to answer questions in a timely manner.

Our team always made a point to be present and available to answer any questions, however after the first survey we decided to dedicate UX designers to the specific pods so the pods always knew who to talk to about specific issues. These designers went to as many scrum sceremonies as possible and made an effort to be present.

I’m satisfied with the overall UX process as it applies to implementing user stories.

Clearly over one quarter of the engineering team thinking we could improve our process warranted some changes. After the firts survey, we started creating UX acceptance criteria to leave as little room for interpretation as possible. We also worked to standardize the approval and handoff process so this was consistent across the nine pods.

I am satisfied with the level of knowledge I have on the end users.

Of all the improvements the UX team made, I am most proud of this statistic. After the first survey showing 84% of the team did not feel like they understood the end user we made several changes.

The first thing we did was implement a “ShopTalk” series where each month we reviewd some aspect of the overall process with the engineers. We encouraged them to submit topics to discuss, and we brought teams from the shops to the office to discuss their experiences first hand. They often brought components so the engineers could see the physical objects to get a sense of scale, weight, and physicallity of the job.

We created what we called a “Mini Brief” for each feature and reviewed them with the teams as often as possibe. The briefs included the overall product goals, the SmartShop design principles, the specific feature’s goals, and a quick overview of the main user persona for that feature. Here are a couple examples of a Mini Brief.

One of the things that I think had the biggest impact was establishing the rule that any future research trip would include at least one engineer. On these trips we encouraged the team to create a daily vlog and post it to the team slack channel. After the first trip it was like a light went off. They got it. And not only did they get it, but they were excited to share with the rest of the team.