The goal of this interview series is to inspire and help people to transition their career into a new or next experimentation related role. In this edition Nils Stotz shares his journey. He is leading the experimentation platform product team at Zalando.

Experimentation needs to be implemented in organizations through a well-thought change management process. I think this will always be the case, even in 10 years.
Nils Stotz
Please introduce yourself to our readers.
My name is Nils and I am leading the experimentation platform product team at Zalando. Before that I was working in various growth roles and always applied experimentation and A/B testing as a methodology to prove my ideas and convince stakeholders. I am really passionate about experimentation and everything that an organisation needs to consider when moving towards a more experimentation-oriented product development lifecycle. I also publish books, academic papers and LinkedIn posts around experimentation and A/B Testing.
What is your current experimentation role and what do you do?
I am leading the experimentation platform team at Zalando and my team is responsible for the experimentation platform and improving both the capabilities of the platform as well as the usage of the platform in the sense that more teams are using experiments in their product development at Zalando. This means I need to identify the unmet needs of teams working with our experimentation platform and prioritise them to improve the platform for the future as well as developing support and learning mechanisms for teams that want to improve their experimentation methods. This also requires us to think through a vision and strategy for our experimentation platform and determine metrics to measure our success as a platform team.
How did you enter the experimentation space? What was your first experimentation related role? Share your origin story here.
I have a Master Degree in Finance and am familiar with scientific methods and decision-making based on quantitative data. When I started my tech career as an Entrepreneur in Residence I once also worked on a few CRM campaigns and worked with customer.io where it was really easy to create an A/B Test. This was fascinating for me because it is just so powerful to think about different variations of a headline and increase the open-rate by more than 30% or even more. I dug deeper into the topic and came across all the possibilities to use experimentation in product development. I applied this basically in all my career steps and now I even build a platform to enable other teams to work more with experiments.
How did you start to learn experimentation?
When I started there was not as much material as nowadays but for sure the material from GrowthTribe was very interesting to read through but also the blogs from experimentation vendors like Optimizely as well as experimentation platform teams from Netflix, Microsoft and so on. I read a lot about this and at the same time could use it in my daily work as a product manager. Later I also worked with Reforge and watched a lot more talks from famous people in the experimentation space and I also attended respective conferences.
How do you apply experimentation in your personal life? (what are you tinkering with or always optimizing?)
I recently started to post on LinkedIn on a regular basis because I listened to the Greg Isenberg Podcast a lot and it just resonates a lot with me to build more media or at least learn how it works to grab the attention of people on a social media platform. I like to try different variants of posts and post the same content in different formats, at different times and so on. The setup of these experiments is by no means scientific but at least I have the mindset to continuously learn about LinkedIn as a platform.
Aside from this, I play around a lot with my morning routine, my food tracking and my workout routine.
What are you currently doing to keep up with the ever-changing industry?
I think the most interesting discussions still happen on LinkedIn and we had some nice talks there where I also try to contribute. Aside from this it is always nice to see folks in real life and attend one or the other conference and meet people in the experimentation space.
What recommendations would you give to someone who is looking to join the experimentation industry and get their first full-time position?
The same advice that I give to anyone who is looking to get a particular position. Make it a goal to connect with as many people in the industry and ask them for career advice or just chat with them to get their opinion. Everyone I met in the experimentation community is really friendly and nice and even if not, the worst thing that might happen is that they are not up for it and this is also fine. This approach is very powerful and even more powerful when you even share the results of these conversations publicly. This will make it easier to join organisations because they will probably already know you.
Which developments in experimentation excite you? How do you see the field changing in the next 5 to 10 years? What will stay the same? What’s not going to change in the next ten years?
Like most technological advancements, experimentation needs to be implemented in organisations through a well-thought change management process. I think this will always be the case, also in 10 years.
What I am really excited about is how we can use AI for experimentation once we get rid of basic problems such as hallucination and things like this. I always wanted to compare the answer of the real Ronny Kohavi and the answer of ChatGPT with all the material ever issued from Ronny Kohavi (and this is a lot) of a certain question. I am pretty sure the answers will not be that much different and I wonder why this should not also be the case for artificial traffic. AI could replace real users and we can experiment a lot faster and solve the low-traffic problem for many companies. This will then ultimately lead us to interesting questions like “how much do we want to personalise a user experience?” because a lot of experiments will have Heterogeneous Treatment Effects and capturing these and individualising certain features creates more if statements in the code and leads to more maintenance. Those are the two factors we need to compare then: What is the potential uplift and what is the potential increased maintenance cost.
Is there anything people reading this can help you with? Or any parting words?
I really want to improve my “personal brand” and “external communication skills”, so if anyone would like to collaborate on something related to product, growth or experimentation on LinkedIn, I would be really happy to! I also love podcasts and speaking gigs, so please reach out if you see something where I could contribute.
Which other experimenters would you love to read an interview by?
I think there are a lot of unsung heroes in the experimentation space, especially the founders of the Berlin Experimentation Meetup: Marcel Toben and Patrick Gunia but also Konrad Heimpel who can talk more about how to use Experimentation to improve ML Algorithms and which tools to use for this. It would be nice to get their views on the experimentation space as well!
Thank you Nils for sharing your journey and insights.