The goal of this interview series is to inspire and help people to transition their career into a new or next experimentation related role. In this edition Reinier Koolmees shares his journey.
Reinier is Chapter Lead of the Data & Analytics team at Online Dialogue. He lives in The Hague (the Netherlands), also known as the International city of Peace and Justice. The city is located near the Dutch coast. When the weather is nice we can find Reinier at the beach, surfing, cycling, chilling or listening to good music at one of the many beach bars.
Online Dialogue is an agency that helps clients grow through an evidence-based and validated approach. An important part of that vision is to allow customers to work more data-driven and/or data-informed. That’s where Reinier and the data team comes in.
Learn it by doing it and make mistakes (and as much as possible). The right attitude is to learn to experiment by experimenting.
Reinier Koolmees
What is your current experimentation role and what do you do?
In addition to the role of Lead and being a Data Analyst within the team, I remain first and foremost a CRO consultant to our customers. This means that I do much more than just analyze data.
In my role, I primarily ensure direction and that the team functions well, that everyone is happy and continues to work on fun, challenging things, and that with a large degree of freedom. I believe that is an important foundation for any well-functioning team. Only in this way we, as a team and as an agency, can provide the highest quality services with regard to data, analytics and experimentation.
As a Data Analyst, I am an important link in the experimentation programs that we run for small and large organizations. In addition to the consultancy part, you can think of, for example, conducting analyses of experiments and mapping the customer behavior of website visitors in an extensive behavioral study. Working on a behavioral study is a close collaboration between the Psychologist and the Data Analyst at Online Dialogue. The result of this is usually also the starting point for a successful experiment program because we have learned from 1000+ experiments that good research is crucial and that this ultimately produces the most successful optimization strategies.
In addition to the experiments, I also regularly address specific research questions related to data and analytics. Questions that can be about AI, GA4 & BigQuery, about which A/B testing tool suits the customer’s wishes, but also very specific questions about website conversion. “Why has our conversion rate decreased over the past year?” The customer can often only partly explain it. Then it is a good idea to engage an external agency such as Online Dialogue to look at it from different angles with a specialist perspective.
I also enjoy training people on-site at different levels through our OD Academy to become all-round conversion specialists.
How did you enter the experimentation space? What was your first experimentation related role?
At a previous employer, conversion optimization was a side task within my position as a Web Analyst. I think many people started out this way and eventually started A/B testing. In that respect, the field has undergone incredible development in recent years because I can still remember the time when VWO was launched in 2009 (15 years ago this year!).
Not much later we also started using the tool. It was mainly used within that company to test ideas on the homepage because it was simply the most important place on the website. One where various departments had a major business responsibility and various stakeholders wanted to claim commercial places while some wanted to strongly defend those places. So in a sense A/B testing also had the function of keeping out so-called zebras (zero evidence but really arrogant people).
How did you start to learn experimentation?
Online Dialogue really opened my eyes when I started working there and they made me realize how beautiful the field of experimentation is, and how complex it can sometimes be.
Fortunately, I learned everything around experimentation from the very best and I am very grateful to Ton Wesseling, Bart Schutz and all the other colleagues I have worked with at Online Dialogue over the years.
It’s been an incredible journey through data and the brain so far!
How do you apply experimentation in your personal life? (what are you tinkering with or always optimizing?)
That’s a very good question. I sometimes have some small thought experiments, or experiments that are about changing my own behavior. Consider ‘Dry January’, for example. I happen to be in the middle of that experiment right now. Cheers to that!
There are also some other aspects of experimenting that I sometimes use in my personal life. A Bayesian way of thinking, for example. Or being aware of the presence of certain biases derived from scientific knowledge from psychology. Or understanding that as a human being you continuously (and largely unconsciously) participate in several, perhaps tens to hundreds of experiments per day!
What are you currently doing to keep up with the ever-changing industry?
Visiting conferences at home and abroad, following thought leaders on Linkedin, following newsletters, listening to interesting podcasts and reading books on various topics that are important to keep up with the field. In addition, it is of course the conversations with my direct colleagues, customers and many other professionals..
What recommendations would you give to someone who is looking to join the experimentation industry and get their first full-time position?
First go for some training to get the basics right, then get inspiration from thought leaders in the market and start experimenting. Learn it by doing it and make mistakes (and as much as possible). The right attitude is to learn to experiment by experimenting.
Which developments in experimentation excite you? How do you see the field changing in the next 5 to 10 years?
It is of course very obvious to mention AI, but I would also like to say that if maturity were to continue to grow in the next 5 to 10 years, and organizations adopt a more integrated approach (by also fully testing with e-mail, in apps and also with code to significantly improve the speed and performance of the website, for example), there may also be a greater need for a Center of Excellence (CoE) to support the organization and the teams even better.
Especially when experimenting becomes more of an automated and democratized process, and therefore becomes part of a corporate’s culture, coaching organizations of different sizes and its people becomes increasingly important in my opinion. This is now only available to large businesses I think. This might be wishful thinking on my part, because I also certainly think that support from AI will really become an integral part of our daily work. But an AI simply cannot take over all the work, although I think that an AI would actually be a good fit for a service providing unit such as a Center of Excellence!
Predicting the future remains difficult and past results provide no guarantee for the future. Certainly not if these have not yet been implemented 🙂
Is there anything people reading this can help you with? Or any parting words?
Thank you for this opportunity! You guys are doing a great job!
Which other experimenters would you love to read an interview by?
There have been quite a bunch interviewed already by you. I would say Joost Baalbergen, Chapter Lead Psychology at Online Dialogue. Based on his expertise, he can tell cool things about his work and share his experiences with the community here.
Thank you Reinier for sharing your journey and insights with the community.
