Hey @alvinkatojr Thanks for the thoughtful questions. I’ll do my best to answer them.
To start, you’re so right…convert and optimizely are powerful but can feel expensive for smaller teams. Affordable (or free) options include platforms like GrowthBook (open source) – lets you run experiments with your existing analytics stack and PostHog which is an open-source product analytics tool with experimentation built in.
That being said, if youre just starting off consider forking the feature flag platform that might already exist where you work OR even a manual “split” test (two different versions of a user base, directing traffic in a manual way) is a good starting point. The key isn’t the fanciest platform but instead, designing a clean test and learning how to interpret results.
As for the phobia question. This is so real. I’ve seen teams hesitate to test because theyre used to simply shipping straight to production and consider that a “win” in itself. The engineering and product culture should ideally be open to insights, if theyre going to ship the feature regardless and arent a metrics-focused organization then..it’ll be hard to shift towards an experimentation driven company. I think the best advice here is to do somewhat of a skunk works effort where you run an a/b test to illustrate the value proposition, and if you can tie it to a feature that product has major interest in…more likely to be successful.
The number crunching can sound intimidating (p-values, power analysis, null hypotheses), but much of the heavy lifting is built into most A/B testing platforms (if it exists) OR you could rely on a data scientist for this area. That being said, statistical literacy does help, but you don’t need a PhD to make sound decisions.
Lastly..thats so interesting regarding my name translation. I learned something new!