The goal of this interview series is to inspire and help people to transition their career into a new or next experimentation related role. In this edition Stewart Ehoff shares his journey. He is Head of Experimentation @ RS Group.
The best thing you can demonstrate when interviewing or searching for new roles is curiosity. If you are inately curious, and have a thirst to learn, the hard skills that can seem complex or daunting like statistics can be taught and learned over time.
Stewart Ehoff
Please introduce yourself to our readers.
Hi! I’m Stewart Ehoff, I’m a software engineer turned experimenter. I’ve been in experimentation for just over 6 years. I live in the United Kingdom.
What is your current experimentation role and what do you do?
My official role title is ‘Head of Experimentation’. In simple terms, I lead the experimentation Centre of Excellence at my current company (RS Group), a near 100-year old B2B global distributor powering industry. My work primarily revolves around our Customer Experience offering in our digital channels, which makes up approx 65% of the company’s revenue.
My job is to help RS make better decisions by leveraging the power of experimentation. I am accountable for the success of our global experimentation efforts, how many tests, how many teams are testing, how accessible experimentation is, the technology that sits behind it, and our value attribution modeling.
When I first started the Centre of Excellence over 4 years ago, it was just me! Now we’re a flourishing team of 6. These days, I don’t sit in the detail of experiments being shipped – my incredible team handles that. This affords me the time to focus on the continued proliferation, education and evangelism of experimentation across the business. This might be building business cases for continued investment & scale, supporting the onboarding and enablement of new parts of our organisation into experimentation, collaborating with the team to build new practices, methodologies and workflows for our global experimentation efforts, or presenting internally & externally to help others to harness the power of experimentation in their work. No day is the same!
How did you enter the experimentation space? What was your first experimentation related role? Share your origin story here.
My formative years were spent building ecommerce websites agency side as a software engineer. I always had a love and passion for data, and often wondered about the impact of my work when I had clients asking for random features, or requests to ‘make the logo bigger’. I ended up pursuing a role as an ‘Optimisation Developer’ at RS nearly 6 years ago. I didn’t know anything about A/B testing, but the concepts sounded fascinating and it was a great bridging role to finally get an understanding of the impact of my work.
How did you start to learn experimentation?
Most of my experimentation knowledge was gained in my role as an Optimisation Developer, but was supplemented with external studying. I spent a lot of time learning about the underlying mechanics of a/b testing, in particular, I was fascinated by bucketing/randomisation methodologies, and different statistical approaches that could be taken in testing.
As I started to build a broader understanding of how experimentation concepts could apply to broader business challenges, the book ‘Experimentation Works’ by Stefan Thomke was a hugely influential piece that unlocked some higher strategic thinking. The stories here of highly successful experimentation companies made me reflect deeply on the possibilities of what experimentation at scale could mean for RS.
How do you apply experimentation in your personal life? (what are you tinkering with or always optimizing?)
Two things come to mind:
- My work from home setup. I’m always playing around with ways that I can optimise how I work. This might be new hardware (for example, I recently upgraded to a fully docked station for my laptop so I can work off two big screens), or software to help me manage and organise my work.
- Physical Health – I have been on a fitness journey for approximately four years, and have experimented with different workout schedules (mornings vs evenings), routines while at the gym, and supplements. I’m recently experimenting with ‘Creatine Gummies’ as I often forget to take it on off-workout days!
What are you currently doing to keep up with the ever-changing industry?
I try to get out of the bubble of my own business as much as possible. Attending conferences in person is a great way to do this, but LinkedIn can often be a great substitute. I am still trying to find my voice on Linkedin, and often am more of a commenter than a regular poster (I post <once a month!), but it’s a great way to keep on top of the industry as it evolves, and learn new things.
What recommendations would you give to someone who is looking to join the experimentation industry and get their first full-time position?
Firstly, you’ve chosen a hugely exciting field! There is so much great material online these days for self-study, much of it free, so getting started has never been easier.
There are plenty of parallel disciplines that sit adjacent to experimentation that make it an easy transition. I started from a software engineering angle, but you could also come at it from Product, Analytics, UX, Marketing, Project Management – there are a lot of transferable skills!
In my opinion, the best thing you can demonstrate when interviewing or searching for new roles is curiosity. If you are inately curious, and have a thirst to learn, the hard skills that can seem complex or daunting like statistics can be taught and learned over time. People just want to work with other pleasant people, so if you’re kind, polite, ask good questions, and maintain a level of curiosity, you’ll be leagues above many others.
Which developments in experimentation excite you? How do you see the field changing in the next 5 to 10 years? What will stay the same? What’s not going to change in the next ten years?
Right now, I am highly curious about two things:
- How ‘Product’ and Experimentation are coming closer together. Experimentation is too often sat in a silo in many organisations, and should instead be considered a methodology for how companies decide what to build, and validate their solutions. I think these silo’s are starting to break down and more companies are beginning to adopt this way of working, but it still feels like a slow burn!
- Warehouse Native experimentation. A lot of experimentation platforms and vendors only really work on digital metrics. Working in a B2B organisation with a lot of offline activity, I’ve built a broader appreciation of business success through offline metrics. A warehouse native approach can help companies to see the bigger picture of their customers’ experience beyond just the website. I am curious about some big players in this space who are starting to build warehouse native capabilities to help experimentation teams solve this problem.
Beyond this, there is a lot of talk about AI at the moment. While there is some amazing capability for AI to augment the work of a human, I don’t think it’s quite ready to steal our jobs just yet. Most companies lack the underlying data structures and cleanliness to be able to truly leverage the capability of AI. That being said, I am keeping a close eye on how it can help us as experimenters to achieve our goals.
One thing that I am certain will not change is the need for experimenters. The bar for digital experiences is still very low, and many companies still operate feature factories, with little to no attribution, and have no knowledge of experimentation and how it can help them to grow their business. When I started my experimentation journey in 2018, I thought I was ‘late’ to the field. What i’ve come to realise is that though there are titans of industry who have been around for 20+ years, there are always newcomers arriving, and that creates lots of opportunities for those of us who work in the field.
Is there anything people reading this can help you with? Or any parting words?
I am always looking for book recommendations! Not just related to experimentation, but business as a whole, and personal growth/development. Hit me with your recommendations!
Which other experimenters would you love to read an interview by?
This series has already covered so many great experimenters! But here’s some I spotted who were missing that I’d love to hear/read about:
- Ellie Hughes
- Mark Pybus
- Natasha Senior
Thank you Stewart for sharing your journey and insights.
