VIDEO: Fireside Chat with Willix Halim (SVP Growth at Freelancer.com)
We had the honour to welcome Freelancer.com's Willix Halim to the studio. We talked about what it's like to build a great technical team, what a growth sprint is and how best to execute it, and growth strategies from some of the big Silicon Valley tech companies.
Q: So where do you think that you made the most impact in your first year?
A: Finding bugs!! So you would look at the data and be able to see immediately that you weren’t charging as much as you had previously told people, “This is how much we are charging.” So I went on to do a lot of bug checking, I also did a lot of querying to make sure the data was there and that the data integrity was there. As a result, I found a lot of bugs! I think then, I just went on to fix those bugs and the things that just popped up.
And this is pretty much common sense but I tried to put buttons on every single page, as a lot of the pages didn’t even have buttons. So I’m like, what is the purpose of this page if there is nothing for you to do? So I went on to just put the correct buttons in the right place. An example would be in the dashboard I put the deposit fund button, AB then went on to test it and it had achieved a 7% increase in the deposit value.
Q: There are obviously a lot of things that affect revenue, how do you stay focused on the things that matter?
A: Good question. I think revenue is a function and that’s exactly the way I see it. So from a mathematical perspective revenue is really when you can build a regression analysis of revenue. It is a function of independent metrics that you can manipulate so things for example in our case the projects that are posted, wanted, completed, how many people verified their credit cards and so on. You can literally focus down on six or seven items that really matter for revenue and then you can just look at those metrics on a daily basis - when one of them goes down then you just start tracking more granularly.
I think the way that freelancer.com works is that you have set product teams. So we have a lot of product teams and every team is an autonomous team who essentially performs a specific goal and focus. Each team will share one to three North star metrics. Although, the team is autonomous in the sense that it will consist of one product manager, one data scientist, three engineers and they are all working towards the same goal. They are moving autonomously, therefore they don’t have to rely on any other teams. So North star metrics are extremely important.
So Freelancer is very transparent, everyone knows exactly why they are doing a job, everyone knows what metrics signify whether or not we are doing a good job and every single member in the sub team knows exactly why and how they are doing a good job and I think this is the most important thing to know!
So a bit more on the ‘teams’. We have a matchmaking team that focuses on ensuring that when a project gets posted we are able to find the best candidate to complete the task either through machinery or other methods. When that team shares the same vision it is so easy to find the North Star metrics; it’s essentially match-making. The North star metrics therefore ensure the project gets awarded and completed, but more importantly completed by the right person.
Q: So now we actually refer to it as growth engineering but that is a term I think that you prefer because it is more of a build and test method. Can you explain a little bit more about that?
A: Sure. So I really don’t have any problem what term people use, but as a public company growth hacking sounds more dodgy than growth engineering, so that’s where it came from. But really the idea is in the mindset, the principle of using data to essentially drive business decisions and more importantly to be persistent about it. I think this is the issue, a lot of the time conventional companies don’t actually think about it that way. It’s about testing a lot faster, learning a lot faster, experimenting a lot faster.
Every quarter we have a concept called growth month, where most of the teams will go and perform numerous tests because studies have proven that the number of tests you do correlates with the metrics or the revenue of that month. But most importantly it’s the principle of testing iteratively and trying to find the fastest way to test based on the data.
An example of this would be: one of the biggest wins of Freelancer.com initially was (and this has become the insight and the principle until now, so we find a huge correlation) this concept of correlation and causation. So a metric can be correlated with another metric but that doesn’t necessarily mean that that particular metric caused the other metric to increase. It is therefore a lot easier to find correlation variables, you will be able to see that metric A is correlated with metric B. But with the power of the internet it is also a lot easier to AB test that to prove causation.
So we go on to AB test it. On the control we would do a lot of tricks to make sure that the first bid comes in within the first five minutes, that way you are able to see the award rate increasing by 40%, significantly on that. This is because of a causation.
When you start with a correlation which is a ‘hypothesis’ so to say, and you AB test that hypothesis things suddenly become clear. The revenue doubled as a result of the testing in a matter of weeks.
Initially average projects took between the post and the award about one week, we shortened that to one day, then twelve hours and you can literally see the LTV (the life time value) of our users increasing significantly.
Q: So is it enough to have that correlation or do you need to prove the causation as well?
A: Yeah so correlation is really easy to find, you just put in the [14:01 tools]. Causation is where you hypothesize and you AB test it.
You find a lot of tactics to make sure you can manipulate these metrics to go up, on the assumption that the metrics that you think will cause it to go up, will go up too.
I think it really drills down to one word: FOCUS.
So I experimented with this as well, I do a lot of experimentation with processes, organizational structure, so the guys who work with me all understand how painful it is to work with me because I will move people around a lot of times, and I’m so sorry for that! But putting people into a room definitely creates a lot of focus, you just see things move so much more quickly.
And I think the HTT drills down to one thing which is, focus on two or three specific metrics at a time to ensure they align with the same goal and that you get the same results.
A: And what is HTT for those of us that don’t know?
High temple testing. So what is funny in ‘growth community’ is we go on and do our own thing. Some companies go and do their own things independently and yet somehow we arrive to the same truth. Right, and of course the name. So I call it growth spring, Sean Ellis calls it HTT, 500 startup calls it – I don’t actually know what they call it, but it is interesting because everybody is sort of experimenting and then we all arrive at this same truth and say oh okay we should do it this way. People call it differently but essentially it is; growth spring or high temple testing. You have to do a lot of tests and you have to iterate a lot faster.
So the idea of Growth Spring is essentially to test to see what things you should be building on in the right way. So instead of building everything right, you have to build the right things right if that makes sense? So the first month is to test what do you need to do over the next two months so that you can do the right things right?
Q: Can you tell us a little bit more about the pirate metrics?
A: Ah yes the pirate metrics. I think it pretty much would classify the funnel of literally every internet user life cycle, so it stands for AARRR. “Aarrr” equals “pirate metrics.”
So it starts with Acquisitions. Acquisitions are about how someone visits the site and then activation is about asking how do you activate them? So at Freelancer.com it is posting a project. Then you come up to retention and whether or not they will come back and use your product. Revenue and whether you make money off them and how you monetize them. Referrals: once they like it whether they will go on and refer more people to use your products. And then at the end after you stay dormant there is resurrection, so how do you resurrect them to use your product?
And you can essentially label or model this user lifecycle to literally any internet products that you have. And that is a good abstract metric to always look at. And there is a good funnel to look at because you will be able to see where the leaky bucket is as well.