The goal of this interview series is to inspire and help people to transition their career into a new or next experimentation related role. In this edition Deborah O’Malley shares her journey. Deborah is many things, founder of GuessTheTest.com for example. You can contact her via deborah@guessthetest.com
Deborah O’Malley
I see life as one, big ongoing experiment. In my work, my relationships, my health, and within myself, I strive for continuous optimization.
What is your current experimentation role and what do you do?
Hi, I’m Deborah O’Malley. I’m a top A/B testing influencer and recognized as a leading women CRO specialist.
I’ve founded GuessTheTest, an experimentation resource dedicated to sharing A/B test case study knowledge and insights. I also run the award-winning optimization agency, ConvertExperts, where I take a data-driven approach to help clients test and optimize their sites.
I’m a published author of six peer-reviewed journal articles, and I currently teach analytics at Queen’s University’s prestigious business school. I’m also currently developing a course for LinkedIn Learning which should be out in the fall of 2024.
When not doing and speaking about experimentation, I can be found in the kitchen making homemade fudge or burning it off at the gym.
How did you enter the experimentation space? What was your first experimentation related role? Share your origin story here.
Without realizing it, I’ve always been into testing and experimentation.
When I was about 8 years old, I unknowingly ran my first optimization experiment!
For a grade-school science experiment, I cut out different shapes and colors of construction paper, posted them on a board, and asked people which version they saw first.
Fast forward many years later, I ended up earning a master’s degree with a specialization in eye-tracking technology.
My research work looked at how to best design attention-grabbing ads. The findings have been published in six peer-reviewed journal articles and are a source of inspiration to digital marketers worldwide.
Upon completing my master’s, I started doing User Experience (UX) testing with a local UX firm.
As part of my training, the firm recommended I check out a resource called WhichTestWon.
So, I dutifully signed up and started taking the weekly A/B tests. Little did I know, this small act would change the course of my career…
Years later, WhichTestWon advertised a position opening for a senior content creator.
I was lucky enough to land the job!
And, in doing so, I moved from eye tracking and UX into the incredible world of CRO and A/B testing.
Here, I knew I had finally found my place.
I loved the whole-brained approach of using data to make evidence-based decisions while still applying creativity to design and test new variants.
Although WhichTestWon was a great experience, there were many bumps in the road.
At its height, the company was acquired and rebranded and soon after, completely dissolved — leaving marketers in a larch.
It was devastating for many. I saw it as my duty to fill the gap.
So, GuessTheTest was born.
As the founder of GuessTheTest, my aim is to provide experimenters with valuable A/B test case study data, and helpful resources to inform, inspire, and validate testing success.
Having produced, analyzed, run, and published thousands of client A/B test case studies, I’ve gained a rich and unique perspective on what works and doesn’t in CRO and A/B testing.
And I’m driven to help teach and share that knowledge with other optimizers.
How did you start to learn experimentation?
Bridging the knowledge from my master’s degree and background in eye tracking and UX testing, I was able to jump in CRO quite quickly.
I was also fortunate to get learn hands-on from one of the industry’s best, Anne Holland. She taught me the ropes and gave incredible insight into the world of testing and CRO.
I also cut my eye teeth making mistakes and questionable assertions through publications on WhichTestWon. OGs, like Peep Laja and Craig Sullivan, would tear apart incorrect statement across social media channels.
It was devastating to me, but provided for quick learning as I never made those same mistakes again!
Gradually, over time, working in the industry, working with clients, and doing a ton of research myself, I started to carve out correct data-driven practices for an effective CRO and testing program.
However, I’m the first to admit, I’m always still learning and have much room to grow.
How do you apply experimentation in your personal life?
I see experimentation as absolutely integral to everyday life, and actually wrote a LinkedIn post about the parallels. You can check out that post here.
On a personal level, I see testing as parallel to snowboarding – which is a long-standing passion of mine.
The reasons why are because both are an iterative process, require data-driven decision making, provide risk and reward, require adaptability, and can be finessed with different techniques. You can read more about my take on the parallels here.
I also like to “A/B” test foods: everything from the optimal Halloween candy, to optimal cooking practices, and which Christmas cookies Santa prefers.
Basically, I see life as one, big ongoing experiment. In my work, my relationships, my health, and within myself, I strive for continuous optimization.
What are you currently doing to keep up with the ever-changing industry?
The industry is changing SO quickly, and it’s becoming increasingly harder to keep up.
With AI taking center stage and Google Analytics transitioning to GA4, my recent focus has been in staying abreast of the changes happening in these realms.
I subscribe to, and diligently read, several AI newsletters, including AlphaSignal, and MindStream.
I also follow GA4 leaders, like Dana DiTomaso, Mia Umanos and Loves Data, to find out the latest updates and changes happening within the GA platform.
Ultimately, I think it’s becoming increasingly harder to be a solo practitioner because one now needs to be so versed in so many different disciplines.
But with a keen appetite for continuous optimization and learning, it’s also an exciting time as the landscape is so rapidly changing.
What recommendations would you give to someone who is looking to join the experimentation industry and get their first full-time position?
I often have newbies contact me through LinkedIn and ask me what they should do to get into experimentation or get their first job in the field.
I think there are 4 important steps to follow:
- Be active on LinkedIn: follow recognized industry leaders, post your own content, and establish a voice in the community.
- Join several Slack channels: there are some great experimentation Slack channels. Join them, participate in the discussions, and keep an eye on posted job opportunities.
- Consume as much knowledge as possible: use sites like GuessTheTest and CXL to take in as much knowledge about CRO and A/B testing as possible. Draw inspiration and knowledge from these resources.
- Subscribe to great experimentation newsletters: take in even more knowledge, and find out about job postings, through great experimentation-focussed newsletters, like Experimental Mind.
Which developments in experimentation excite you? How do you see the field changing in the next 5 to 10 years? What will stay the same? What’s not going to change in the next ten years?
In the next 5-10 years, I think the field will look very different. I see 3 major trends happening:
1. Industry consolidation
Right now, there are a lot of smaller testing platforms and testing agencies.
Over the coming years, I think there will be a lot of consolidation and merging of agencies and platforms resulting in fewer, larger players.
For example, within the past few years, Dynamic Yield was acquired by McDonalds (and now Mastercard), Adobe acquired Figma and Marketo, Episerver acquired Optimizely, and Content Square recently acquired Heap.
In the near future, I think more of these types of acquisitions will happen. Smaller players will get swept up into enterprise-level organizations.
2. AI will take over
AI is still in its infancy, but will inevitably take over the field.
Eventually, experimenters will rely on AI to do full-funnel data analysis, come up with informed test hypotheses and ideas, and run tests on fully AI-powered platforms.
AI will likely also be incorporated into faucets like Figma wireframing and design, making designing test variants as easy as pushing a button.
While this direction may be threatening to experimenters, with great change will, hopefully, come great reward.
3. True personalization will finally emerge
With AI leading experimentation, I believe “true personalization” will finally emerge in that websites will become fully unique browsing experiences for each individual visitor, based on their own specific user-identified preferences.
So, for example, each individual user coming to the same site will see different pages, images, copy, or positioning of elements, depending on that user’s identified intent, interest, and personal details.
While marketers right now are grappling to provide so-called personalized experiences, I believe AI will, ultimately, transform personalization to a level not yet even imaginable.
Is there anything people reading this can help you with? Or any parting words?
I’d love to know how I can provide more value within the experimentation community, especially through GuessTheTest and on LinkedIn.
Any feedback or recommendations would be welcome.
Which other experimenters would you love to read an interview by?
Would be interested in reading an interview from Linda Bustos.
She’s been actively involved in the field for a long time and is the current 3X return champion on the Can You Guess The Test game show.
I’d love to learn her background story and hear her predictions on where the industry is headed.
Final thoughts: Thanks for this opportunity!
Thank you Deborah for sharing your journey and insights with the community.
