The goal of this interview series is to inspire and help people to transition their career into a new or next experimentation related role. In this edition Margaret-Ann Seger shares her journey. She is Head of Product at Statsig.
Do the thing! This is generic advice beyond just the experimentation industry, but in general I’m always impressed with candidates who find a scrappy way to get experience in the domain they’re applying to. Doing is always > talking.
Margaret-Ann Seger
Please introduce yourself to our readers.
Hi! My name is Margaret-Ann and I currently lead the Product & Design teams at Statsig, the modern experimentation platform for product builders.
Before Statsig, I spent 9 years in Big Tech at Uber and Facebook (before it was Meta). During these years, I worked across a number of product and growth teams, shipping everything from Facebook’s first iteration of video ads to Instagram’s early account suggestion features to cash payments on Uber in international markets.
It was during these years that I developed an appreciation for how powerful good experimentation tooling can be. So when I met our CEO Vijaye Raji and he told me his vision for Statsig, I was immediately excited.
What is your current experimentation role and what do you do?
My job is to work across our Engineering, Product, and Design teams to shape the end user experience that thousands of Statsig customers use to run reliable, trustworthy experiments.
I think this is a pretty cool job- I get to hop on the line with customers and understand their workflows and cultures of experimentation, and then translate that into features that supercharge them. I work through hard problems with our data and engineering teams as we push the scale of our systems to process trillions of events a day.
And I get to partner with our design team on the challenge of how to simplify complex statistical concepts to be accessible to those newer to product experimentation, while still preserving the power of the tool that our most advanced users have come to know and love. Balancing all these inputs day-to-day is challenging, but a ton of fun.
How did you enter the experimentation space? What was your first experimentation related role? Share your origin story here.
I started my career as an experimentation practitioner, in my roles at Facebook and Uber. At Facebook, the internal tooling for experimentation was very good, but at Uber when I joined in 2014 there wasn’t an internal experimentation platform yet. Experiments were run via spreadsheets and tagging riders and drivers into different experiences manually. I fondly remember testing cash payments and other local payment methods in international markets in this “scrappy” way.
In 2015/ 2016, Uber started seriously investing in an internal XP Platform which they named Morpheus, which streamlined much of this and alleviated our poor Data Science team from having to run manual experiment analyses every time we tested a product change.
It wasn’t until I left Uber in late 2020 that I realized most companies still didn’t have access to great experimentation tooling. So when I met Vijaye and he conveyed his vision, the pitch was a no-brainer. I had existed in the “before” and “after” experimentation platform worlds at Uber and knew how much of a game changer good tooling could be here.
How did you start to learn experimentation?
Well, the real story is frankly pretty embarrassing. In my first role at Facebook, one of my first projects was working on an early version of video ads. I was a very green PM and didn’t know what a full product launch cycle should look like. I jumped into the project whole-heartedly and when it came time to test, we launched the product to a few advertisers… except we didn’t run it as an experiment. When the advertisers came back to us asking for lift measurement on the videos, we realized our mistake. I was mortified.
Fortunately my boss at the time was very gracious and took the hit for me and convinced the advertisers to re-run their ads, but it was a very sobering learning moment. From that point onward, I always ensured that every new feature going out the door had the right logging, was being run as an experiment when appropriate, and generally had good observability so that I never got caught in a situation like that ever again.
How do you apply experimentation in your personal life? (what are you tinkering with or always optimizing?)
I have a 7 month old son which is the mother of all experiments, right? Jokes aside, I really have found motherhood to be a giant experiment in a sense. Nobody comes in having done it before with their first and you’re constantly trying things to arrive at what seems optimal for your child. I went through a phase of A/B testing diapers to minimize leaks.
Now we’re trying a slew of new solid foods to test whether he’s allergic to anything.Unfortunately the feedback loop is slow and imperfect. Is he crying due to gas from what he just ate or is he teething? Could he be sick? Maybe all the above! I tell myself that every generation before me has successfully tested their way to success here though, so with enough trial and error we’ll get there. 🙂
What are you currently doing to keep up with the ever-changing industry?
I love learning from our customers and our team. We actually co-design many features in tandem with our customers because we’ve found that rooting the product in a tangible use-case and set of requirements guides it to be immediately useful for someone and often these requirements are common across customers. We’re in the process of co-designing Geo Lift testing with a customer right now and we’ve done this in the past with modified versions of Sequential Testing and Stratified Sampling.
What recommendations would you give to someone who is looking to join the experimentation industry and get their first full-time position?
Do the thing! This is generic advice beyond just the experimentation industry, but in general I’m always impressed with candidates who find a scrappy way to get experience in the domain they’re applying to. Can you start running experiments in your current role? If you’re in school or not yet employed, can you deploy experimentation somewhere in your life? Maybe you have a personal website or a side hustle and can set up a few different experimentation platforms and provide feedback on the onboarding experience. Doing is always > talking.
Which developments in experimentation excite you? How do you see the field changing in the next 5 to 10 years? What will stay the same? What’s not going to change in the next ten years?
I know this is a cliched answer, but I do think AI is going to drive some interesting innovation in the experimentation space. For example, one of the hardest problems in experimentation- actually in product development generally- is knowing what tests to run. Where are the gaps in your product and where should you be focusing your efforts to improve the experience?
As teams build up a body of historical data, experiment results, etc. I believe that every company will have their own model built on top of their data (something previously only available to the biggest FAANG companies). A mix of ML and LLMs will enable an experience where your tooling informs you proactively when a metric is trending in the wrong direction and suggests mitigations, or actively points out gaps in your metrics that you should be targeting with product solutions. Data will go from a pull model to a push model, speeding up time-to-impact and democratizing who can engage with it It’s all part of the broader trend of empowering fewer people to be more full-stack builders, and I see this trend as inline with Statsig’s mission rather than a threat to it.
What won’t change? The need for data. Data will become more important than ever, because it will be the basis of this company-level product building model that will inform every aspect of the roadmap, based on real user behavior inputs.
My guess is the role of PM, engineer, and data scientist will merge into one and these generic “product builders” will be responsible for leveraging this product building model to mine insights, turn those into product solutions (idea -> design -> code), run an experiment, and repeat. Of course, “taste” will still be a part of this process in terms of refining the proposed product solution, and I could see a world where taste will become a critical skill companies hire for over time.
Is there anything people reading this can help you with? Or any parting words?
Give Statsig a try & let me know what you think- I love hearing from customers directly!
Which other experimenters would you love to read an interview by?
If you could swing an OG growth leader from Meta, like Alex Schultz or Javier Olivan that would be very cool.
Thank you Margaret-Ann for sharing your journey and insights.
