Distyl
ROLE
Product Designer
TIMELINE
Oct 2024—Dec 2024
3 months
TEAM
1 PM
2 Product Designers
1 UX Researcher
5 Software Engineers
CLIENT
Edward Chew
(Design Engineer @ Distyl AI)
THE MISSION
The current golden set management process for busy AI specialists at Distyl was fragmented and difficult to navigate.
Our goal was to create an application that displays the data for easy comparison between the expected outputs and current outputs to improve model improvement practices.
SOLUTION
Opening a Cell to See Current vs. Expected Output
Version History of a Single Cell
Selecting Multiple Data Points of Assign
Opening Activity Page
RESEARCH
The current space was not dynamic enough to keep up with the specific use case of comparing LLM responses.
We kicked off the project with in-depth interviews with AI Strategists, Data Scientists, and Senior Engineers to map out workflows, frustrations, and ideal solutions. We realized most Distyl employees rely on general spreadsheet tools, such as Excel, Google Sheets or Notion.
PROBLEM SPACE
When working with large datasets in Google Sheets, it’s difficult to identify key differences during comparison.
Spreadsheets lacked flexibility, it is difficult to update multiple datasets of the same type simultaneously.
It’s challenging to align on evaluations to maintain a consistent shared understanding of the data among AI strategists and their customers.
White boarding and thinking out-loud exercises
Taking our findings, we began white boarding the problem space and group key overlapping pain points in order to clearly define solutions. With some of us being new to AI terminology and technology, aligning our understanding of the problem with the right solution space was a challenge. As a team of developers and designers, we supported one another’s knowledge gaps!
DESIGN ITERATIONS
Incorporating feedback from stakeholders
Our MVP designs didn’t exactly solve all the pain points we found through our discussions with AI strategists. While our MVP’s provided a solid foundation, feedback from Distyl highlighted opportunities for deeper collaboration tools and more flexible data management options.
LEARNINGS
I started with an ambitious vision for what this product could be. But as our timeline shifted, I had to pause and rethink about what actually mattered most to the people using it. I scaled things back into an MVP that was focused and realistic. That shift forced me to really differentiate between user needs and nice-to-haves. Once we locked in the essentials, I was able to build forward again, this time with more clarity.
Before diving into this project, I was still new to AI workflows and terminology. I quickly realized that to design something truly useful, I had to understand our users’ process inside and out. That learning curve was steep and required a lot (like, a lot a lot) of questions and interviews—but in the end, it proved essential.
