Building and scaling a configuration portal for a developer platform

Creating a self-service onboarding portal to replace a white-gloved onboarding process for thousands of dev teams with complex code releases.

At a Glance

My Role: Lead UX Designer: Interaction & Visual Design, Research, Facilitator Prototyping
Timeline: 4 Months
Team: Immediate team: 1 PM, 4 developers, Secondary team: 2 PMs, 10 Developers,

Problem

Azure Build Health, a magical dev platform that reduces the risk of bad code being pushed, had to whiteglove onboard teams onto our code platform manually.

Goal

Craft an onboarding experience that's flexible enough to support our customers different data pathways and pipelines so we can make team's onboard themselves instead of bothering us all the time to do it.

Impacts

Teams on platform go up (40-> 1500 in 6 months)

Time for team to onboard to the platform from 1 month -> 3 hours

Onboarding Landing Page. Many objects. wow.

See bugs. Smash bugs.

Set up custom views to see what you want with big brain logic so it auto updates.

Smart Computer Wizard People Want to let other Wizards Add Their Tests

Wizened future wizard travelers of Azure Build Health (ABH) ask Me, rectangle and diagram man, to help give their users, other wizards, the ability to add their test pipelines without bothering ABH Wizards. Me oblige. Wizards wanted the portal to let users manage two things:

Test Pipelines

1337 hacker wizard teams test their build before launch, and create more. Right now wizard teams require our wizards to support them. Help them do update their work.

Test Pipelines

1337 hacker wizard teams test their build before launch, and create more. Right now wizard teams require our wizards to support them. Help them do update their work.

Test Pipelines

1337 hacker wizard teams test their build before launch, and create more. Right now wizard teams require our wizards to support them. Help them do update their work.

Test Pipelines

1337 hacker wizard teams test their build before launch, and create more. Right now wizard teams require our wizards to support them. Help them do update their work.

Release Views

Some wizards only care about their magical experiments, release views let them only see their magic. Me no understand the intricate details, me only know rectangles and product value.

Release Views

Some wizards only care about their magical experiments, release views let them only see their magic. Me no understand the intricate details, me only know rectangles and product value.

Release Views

Some wizards only care about their magical experiments, release views let them only see their magic. Me no understand the intricate details, me only know rectangles and product value.

Release Views

Some wizards only care about their magical experiments, release views let them only see their magic. Me no understand the intricate details, me only know rectangles and product value.

Gotta go fast

Scope is 2 weeks for project outcomes, short term focus, no looking at long term,

I tricked the wizards into doing work for me by drawing diagrams so I can know their secrets, and understand the big picture, even though thats not the scope. Some call this democratizing design, I call it outsourcing.

I styled on the PM by showing them how to draw a diagram 4 real. Look at that subtle off-white background. Lets see Paul Allen's diagram.

Do It Again

1 month later, Big Boss Man EVP wanted all of Azure's bedrock teams of Azure onto our magical platform code catching platform , which meant we needed to add more onboarding features to let user manage:

Build Pipelines

These magical portals transport the code, and occasionally new ones need to be added.

Build Pipelines

These magical portals transport the code, and occasionally new ones need to be added.

Build Pipelines

These magical portals transport the code, and occasionally new ones need to be added.

Build Pipelines

These magical portals transport the code, and occasionally new ones need to be added.

Bugs & Incidents

Stinky issues need to be associated with the correct build pipelines without needing to manually associated

Bugs & Incidents

Stinky issues need to be associated with the correct build pipelines without needing to manually associated

Bugs & Incidents

Stinky issues need to be associated with the correct build pipelines without needing to manually associated

Bugs & Incidents

Stinky issues need to be associated with the correct build pipelines without needing to manually associated

I am SPEED

No time to explain (2 week timetable, can't test with lots of customers), only time to design and talk to 2-3 customers. Our customers were all using release views so I put that as the first entry portal before seeing their tests.

So many tests. SO MANY TESTS 😱

Suffering From Success

The new onboarding hub was constructed during December, at the same time as the annual
❄️BIG CODE FREEZE ❄️. During the freeze there were far fewer Azure outages, highlighting the importance of ABH. Overlord EVP of Azure, Scott Guthrie decreed that he wanted 1500 services, up from the 44 currently, to be onboarded onto the platform in 6 months.

Hol' up, Wait a Minute, Something Ain't Right

My party of wizards and planners wanted to do another sprint. Instead of rearchitecting each time, I pumped the breaks for a bit and went back to understanding each part of Onboarding, so that we could have a solid base so if additional features asks came, there would be less dev and product work in the future. This involved leading three workshops and diagraming sessions with the wizards.

Witchcrafting with Wizards

Many thanks to my wizard companions (Mitch and Cecilia) to demystify and scry out all future needs, where we bounced ideas off each other to come up with a unified portal.

Paradigm Lost and the Interaction Journey

Conducting more user research we found that the Release View containing the Build Pipeline introduced friction, so I swapped the hierarchy to be pipelines first, and for teams with more complex requirements, they could add a release view on top of a pipeline. Then I used the dark magic to craft a variety of rectangles.

Above & Beyond (On a Good Day)

While working on this project, I included several recommendations to THE COUNCIL for further improvements to guide the product once I journeyed on.

Integrated Flows at the Pipeline Page

I integrated Release Views, a new concept, into other Azure DevOps build quality tasks which start at the pipelines page.

Onboarding again and again and again and again (ad infinitum)

Onboarding new test pipelines was a common task but out of scope for this work. I built this workflow into the main ABH page, which let users avoid going to the onboarding hub, reserving it for more heavy technical tasks.

Validation for Now and the Future

I tested the designs with several teams inside of Azure to make sure the new designs supported all of Azure team's data pathways and pipelines. I also went solo on conducting some guerilla testing with teams outside of the Azure org and validated what additional features they might need for the platform to be optimized for them..

Onboarding Hub.

Queries track issues related to each new build that passes through a pipeline, requiring only a single-time entry.

Set up custom views to see what you want with big brain logic so it auto updates.

Retrospective

Working closely with my party of wizards, clerics, and compatriots. I was able to navigate this complex world to guide the product to a solution that works for a large group of teams with different complex data pathways and pipelines. Thanks to my wizard counterparts helping me learn how to navigate the abyssal magics thanks to their aid.

Sometimes you solve for the edge cases, sometimes you solve for the golden path, or the simple path in this case. Sometimes its better to optimize for Graceful degradation and other times progressive enhancement is the ideal.

Lead PM of Azure Build Health: