The goal of this interview series is to inspire and help people to transition their career into a new or next experimentation related role. In this edition Dave Masters shares his journey. He is Senior Director of Product Management at Realtor.
It has never been easier or cheaper to test ideas – specifically in web applications. Between no-code, all of the AI tools that exist, have made it pretty easy for anyone to build something cheap, to test a concept with real people.
Dave Masters
Please introduce yourself to our readers.
I’m Dave Masters, and have worked in product management for nearly 2 decades, in a wide variety of roles across the product function (which I love!). Additionally, I’ve done a bit of mentoring for aspiring product managers on the side.
I’m based in the NYC area, and outside of work, I spend my free time with my family, traveling, skateboarding with my 8 year old son and making music!
What is your current experimentation role and what do you do?
Currently, I’m a senior director of product management for Realtor.com. Our team oversees a range of products that span from new products and growth areas, to optimization of well established products. As you can imagine, the testing for those two dynamics are quite different, which is really interesting to me.
How did you enter the experimentation space? What was your first experimentation related role? Share your origin story here.
While maybe not directly what we refer to as “experimentation” a product management role is all about experiments. When I first started working in product, I worked on tools supporting our internal operations. I would notice how the systems were used, and look to optimize for efficiency. The way I learned to build products, do research, write all the requirements, and then build the thing. Sometimes successful, sometimes less so. Over the years I’ve learned that everything I was building was an experiment but was not treated as such. While I had good confidence in the pain points we were solving from spending time with users/customers, we often doubled down on the idea we came up with too early. Over time, I’ve learned there were ways to bring greater confidence that those things you finally do end up building, had a greater shot of success through working in smaller batches and building confidence that we were on the right path. My experience and improvement in the product craft over time helped establish some new ways of thinking. One of the most critical moments of my career was when the team I was working on, was consulting with an innovation firm called Neo. The thing I loved about this arrangement was that they specialized in teaching ways of working to improve innovation and building within your organization – building repeatable processes for after the engagement is over. This challenged the way I thought of things in the past and opened up new ways of working – truly, thinking big, but working small. The Neo team was such a solid group of practitioners, I still greatly admire their work today.
How did you start to learn experimentation?
While I was always “experimenting”, the aha moment for me was when Josh Seiden (Lean UX, Sense and Respond.) was working with our team. We went into a meeting and I shared all the things (I thought) we needed to build. This was a massive list of features. They were all rooted in solving real-world pain points that our team had learned, so I had some conviction, but the list was very long. Josh, paused me; “that is a lot of stuff! what if we can only build one of those things to start, which is the most important?” While I had done both optimizing existing products and also building new things from scratch, I expected that this was an opportunity to build something entirely new and revolutionary. I believed our team had the answers, but Josh was questioning our assumptions. This took me a bit by surprise. I felt so sure about these things, but it took me stepping back and reframing what I (thought I) “knew.” Why did we feel so confident? What could we do to increase our confidence further?” He and the team coached us on new ways of working. Doing product-related work for the prior 10 years, this was forcing me to unlearn how I was doing things to change the way I would need to do things. Breaking our list of features into a series of assumptions, prioritizing the riskiest ones, and constructing cheap tests that we could run quickly to help us gain further confidence in our solutions. This opened my eyes to new ways of working that have changed my view of how I work ever since.
How do you apply experimentation in your personal life? (what are you tinkering with or always optimizing?)
I’ve been making music for a long time, and am constantly evolving ways of thinking on how to create and release music. I have 2 main projects, a recent push since the pandemic (out of necessity to create!) into making solo music, and another with a band I’ve played with for a long time. The band does things a bit more traditionally. We write a bunch of songs and get enough for an album. Next, we go record that album, mix it, master it, release it. That whole process always takes longer than you expect. You get a single crack at it, and it can cost a lot of money. Contrast that with my solo project, which so far this year, I’ve released more than 25 releases (one weekly release) and nearly 50 songs.. What started with a journey of pushing myself to keep creating, has turned into a repeatable process that I’ve established in which I get the opportunity each week to tune how I do things based on any aspect of the process. How can I get my songs to sound better? How can I try different workflows for writing songs? Did this week’s release make it to anyone’s release radar on Spotify? Did my Instagram push this week have more engagement or less engagement based on the different tactics? This is just fun for me, so I don’t put a great deal of seriousness into any aspect of it, but every one of the steps of creating and releasing music has parts of the process that can continuously improve. My curiosity mixed with my desire to create has established a forcing function to experiment with both each week.
What are you currently doing to keep up with the ever-changing industry?
I love reading, listening to podcasts, and learning about new ways of building products. We are so fortunate that there are so many resources for that today. Hearing from others, and getting inspiration from lots of other people, in all industries, always motivates me to keep pushing and taking those themes into my work.
What recommendations would you give to someone who is looking to join the experimentation industry and get their first full-time position?
Think about how your current work has an experimental aspect to it. We have subconsciously experimented our whole lives, but framing it as such requires you to rethink things a bit. So start by examining and tailoring your previous experiences into how they were an experiment, what you learned, and what you did with those learnings. Think about the type of experiments you like driving. Are they the 1% improvement on massively established products, or early discovery ideas for companies that are radically rethinking how things are done? Both of those have very different approaches and thus require you to focus first on different toolsets.
Which developments in experimentation excite you? How do you see the field changing in the next 5 to 10 years? What will stay the same? What’s not going to change in the next ten years?
It has never been easier or cheaper to test ideas – specifically in web applications. Between no-code, all of the AI tools that exist, have made it pretty easy for anyone to build something cheap, to test a concept with real people. There are so many tools, resources, and frameworks now that can help you establish what you need to test for. This has really motivated me to think about how we can leverage that in our work. Given the recent boom of AI, I see this continuing to make it easier for testing, and all of us in the product space will continue needing to hone our craft for new ways of working. What’s not changing – knowing what to test for, and understanding what you need to measure. This is true for every experiment. I think about how David J. Bland shares that while the lean startup movement is constantly referred to as Build, Measure, Learn, Loop, it’s best to think about testing in reverse order. What do I need to learn, so that I know what I need to measure so that I can build something with that in mind. I don’t see this changing.
Is there anything people reading this can help you with? Or any parting words?
I’m inspired by continuously learning and improving. Very grateful for all those who take the time to share their stories, their view of the world, and the evolution of building products. So please keep sharing!!! I’m happy to help the community in any way, so if you feel stuck working on an idea, need help with breaking down an experiment, or breaking into a field in experimentation, please feel free to reach out. If I can’t help, I can possibly connect you with someone who might be able to!
Which other experimenters would you love to read an interview by?
Josh Seiden and David J Bland are two people I have learned a ton from over the years. Additionally, Giff Constable who wrote “talking to humans” and “testing with humans” is a big influence. Not surprisingly, they are all former Neo people.
Michelle Serro, a founder who has built a few different companies, would also have some really great stories for early stage building of ideas.
Happy to connect you with some of the data science folks in our org as well!
Thank you Dave for sharing your journey and insights.
