We hear a lot of discussion around the best practices for launching a mobile game, and one particular topic that often comes up is: whether or not to do a soft launch. And if you do soft launch, how can you tell if your game will be successful? Much of the mobile games industry has taken the tactic of soft launching seriously – in September alone, there were more than 100 games soft launched on the Apple App Store and over 600 on the Google Play Store.
This blog post will explore what a soft launch is and why it can be beneficial for game studios. We’ll also dispel some of the myths that are commonly associated with soft launches and help you figure out if this strategy is right for your game. We’ll also address why there are so many more games soft launched on Google Play (hint: they have more options to support a variety of soft launch strategies).
What’s a Soft Launch?
A soft launch (or sometimes “geo test” or “beta”) is when a game is released to a limited number of players before it is released worldwide, usually by releasing only to specific countries. This allows developers to test different aspects of the game, such as the gameplay, graphics, and economy in a real-world environment without the pressure of a full release. Soft launch enables player feedback and metrics which can be used to compare to benchmarks that both Apple’s iOS App Store and Google Play offer in their dashboards.
There are typically two stages of soft launch: alpha and beta.
- Alpha is the earlier stage and is often used to test whether the core of the game is working, including the technology and core gameplay loop.
- Beta is later and is used to test the meta gameplay loop, marketing acquisition, server scalability, and monetization.
Why is soft launch important to consider? Because you only get one shot at a first impression with most of your potential players and you want to make sure the game has the best possible shot at being successful when you launch globally. Mobile gamers are spoiled by choice and have a low propensity to return post-churn. We’ve seen that typically, it is between 5-10x more expensive to acquire a player who already quit than to acquire a new player.
Additionally, distribution is one of the biggest challenges in mobile games – if your users don’t retain or monetize, you’ll have trouble scaling your game via organic growth or paid acquisition. By releasing the game in stages, developers can get valuable feedback from players that can help improve the game before it is released to everyone.
Myths About Soft Launch
Before we share more perspective on soft launch strategy, we wanted to dispel some common myths:
The country or market chosen for the soft launch will accurately represent how the game will perform worldwide
While it’s important to choose a market that closely aligns with your target demographics, it’s also important to keep in mind that every market is unique and may have different reactions to the game. For example, many developers choose Canada to test how a US launch would perform. In our experience, there are too many variables to accurately simulate a launch in another country, however, retention and monetization metrics compared to other regions can give you some directional understanding of performance.
Soft launch can only be done on mobile platforms like iOS and Android
Though the focus of this post is mobile, a soft launch strategy can also be applied to PC and console games. This can be especially useful for cross platform games to understand the dynamics of players on different platforms, and improve gameplay and performance.
Soft launches are expensive
The investment required for your soft launch is dependent on your goals for soft launch. If you have a strong understanding of your audience, you will be much more likely to target them via organic or paid means in a cost effective way.
You need to soft launch a game for it to succeed
Soft launch is a way to solicit player feedback before a broader global launch, but isn’t necessarily needed to be successful. We believe it is usually better to soft launch, but there are plenty of successful games that went straight to global launch.
Resetting player progress after a soft launch will anger the community
While some players will respond negatively if you reset their progress, many understand that this is the “price of admission” for accessing a game early. As long as you’re upfront with players that the test is limited to a certain number of days, they will understand when the test ends. For example, Supercell recently launched a “Limited Open Beta” to test their game Flood Rush (ultimately based on soft launch Supercell decided not to release globally). If you’re resetting player progress, it is a best practice to refund players who spend during soft launch (or at a minimum, give them the equivalent value of in-game currency to spend later).
What Can you Learn from Soft Launch?
There are a few questions you should seek to answer with a soft launch, though it’s difficult to answer all of these at a high level of fidelity. If you’re resource constrained, you should think about prioritizing your goals based on what’s most important to the team.
Is the game stable/performant?
Building a bug-free game is hard, and while your QA team is heroically finding and triaging bugs, there’s nothing like a soft launch to show you where the holes are. Acquiring users at scale from lower-cost geographies is a great way to stress test servers and also get performance data from long-tail devices. There are a TON of lower-spec Android phones that your QA team won’t have and should probably not be purchasing. Some metrics to keep in mind for this sort of testing include: FPS, crash rates/DAU, crashes/crasher, and latency. Google Play recommends monitoring via their Android Vitals Dashboard.
Is the core game fun?
The most important part of a game experience is the core loop. It’s the atomic gameplay loop that people will experience over and over and should be HIGHLY repeatable. If a game’s core loop doesn’t work, players will never stick around long enough to hit the meta loops that keep them around long term and can generate revenue. Testing core loops effectively requires getting players in soft launch that simulate your target audience at scale, so targeting that audience is important. On the flip side, you can save some energy/money by remembering you don’t need to target audiences that necessarily monetize or worry about how players are engaging with other parts of the game. Some metrics to keep in mind for core loop testing would be:
- First Time User Experience (FTUE) completion rates (Wooga’s game director, Annelie Biernat, recently shared why “Optimizing the FTUE is definitely worth it”)
- D0 playtime & repeat sessions
- D1-D7 retention
- Sessions/day
- Minutes played/day
Does the game have legs?
If your core loop is working, you’ll want to make sure the game is highly retentive. Retention is king, and it’s the most important metric in all of free-to-play. Retention compounds over time and has a large cumulative effect on DAUs. Retention also leads to organic growth – the longer players stick around the more chances they’ll have to tell their friends about the game. Retention also leads to revenue – the more sticky your game is the more likely it is that a player will support the game by monetizing. One key metric to look at is D30 retention, but other metrics are important too including elder system engagement %, DAU/MAU and DAU/WAU. Google Play data suggests 50% percentile for top games DAU/WAU = 55% and DAU/MAU = 31%.
Ideally you want to make sure your D30 retention is strong enough to stand on its own. Testing D30 retention means that your test will need to run at least 30 days – but to get 30 cohorts of D30 retention your test will have to run at least 60 days! Additionally, since a smaller % of each install cohort will get to D30, you’ll need to work backwards to calculate statistical significance for your install cohorts. Testing D30 retention can be difficult for smaller developers that don’t have the time or resources to get highly statistically significant data.
A hack to proxy D30 retention is to look at the ratio of D1:D7:D30 retention – according to Google Play, the Median DAU return is 77%. Looking at the shape of your retention curve can help approximate longer-tail retention.
Will the game generate revenue?
Ultimately your goal is to create a successful business and to do that you’ll need to generate revenue. Importantly, revenue isn’t just about the money – being able to generate revenue reliably helps you re-invest those dollars back into user acquisition which can help your game grow over time. Many successful mobile games don’t make much of a profit in the first few years of operations. Developers choose to invest in growth for two main reasons: first, network effects from a larger audience can lead to more organic growth. Second, older games have a hard time growing – it’s easier to grow when you have the buzz of a new game (and that new game is featured in the app stores!). After non-linear growth becomes challenging, teams have the option value of taking their foot off the UA pedal and focus instead of revenue optimization/profit maximization.
There isn’t a straightforward playbook on how to test monetization in your game because each game has a different monetization system. The most important signals to understand are:
- What % of players convert into becoming payers?
- Once someone pays, how much do they purchase and how often do they pay?
To get a clean read on those signals, you’ll have to make sure that the players you’re testing against are a representative sample for what you’d expect the target audience in a scaled launch scenario to be. Remember that the % of players that actually pay is typically single digits, which means to get statistically significant data you’ll need to make sure you have enough test players. As a rule, you can expect between 2-5% of your players to be payers (2% for casual games, 5% for midcore), and for 80% of your revenue to come from the top 20% of payers.
Can we grow the game?
Finally, if you know you’ve got a strong core game that retains and monetizes, you’ll still need to ensure the game has a path to sustainable growth. The two primary growth categories are paid and organic and both are important ingredients to any game’s growth story. Stress testing paid growth is a bit easier to do than testing organic growth.
There are several components to testing paid growth. Your ultimate goal is to measure and minimize the cost of acquiring players (your “CPI” or “CAC”). To do this, you can conduct paid user acquisition campaigns and experiment with different targeting strategies, ad creatives, and budgets to determine which ones are most effective at driving player acquisition at the lowest cost. You can also track the long-term engagement and monetization of players acquired through paid campaigns to make sure LTVs are above CPIs. If LTVs don’t exceed CPIs you will be losing money on every paid player you acquire!
If your game is multiplayer, we recommend considering launching on both iOS and Android during soft launch to ensure you have a full understanding of how the network effects across platforms influence retention and organic growth. Brawl Stars, for example, was originally launched on iOS and retention increased once it was launched on Android.
Remember, you don’t have to test everything. Choose the testing plan that’s right for your product and timeline. But remember, decide what you want to get out of your soft launch upfront, be clear about the metrics you need to hit to reach the next stage, and be disciplined about holding that bar. The game only gets one shot at a first impression!
How to Plan a Soft Launch
There are a few key factors to consider when planning a soft launch. Here’s a checklist to consider:
Ultimately, your choices will determine how you balance work and overhead for your team vs. quantity and quality of information from soft launch players.
Limits of a Soft Launch
Soft launching can be a valuable tool for game studios, providing insights on the performance and potential success of the game before its worldwide release. But it’s important to keep in mind that there are limitations. By carefully planning and analyzing the results, developers can use soft launches to improve their game and increase its chances for success on a larger scale.
In our experience, there is no way to fully simulate a global launch during soft launch. It is possible to estimate how quickly CPIs in certain regions will saturate and to have an understanding of the shape of your game’s retention curve. However, in soft launch factors like seasonality, network effects across regions & platforms, brand effects of global launch marketing, and platform support are all difficult to fully understand ahead of global launch.
Soft launch should be leveraged to continually improve your game until you are at a point where you can make a go/no go decision. Killing a game in soft launch is difficult, but the option should be on the table if you’re not able to deliver the improvements that you believe are necessary to launch a successful game. Once the game launches the team should be prepared to spend years iterating and improving the game. The passion needs to be there as do the metrics. You will also likely have to scale the team to support ongoing operations including live ops, player support, community, and marketing. Is this a game that you would bet your company on?
Once you are committed to launching, soft launch helps you define your risk tolerance. If based on the metrics, you believe you have a strong product market fit, leaning into launch investment could make sense to maximize the golden cohort. A word of caution, however, if you do not have an understanding of your longer term retention curve (90 days +) you may have later game issues that you have not yet identified. We generally recommend building plans that are flexible and can quickly scale up or down based on performance.
Conclusion
We are big believers in getting games into the hands of players as early as possible and iterating based on feedback. To that end, soft launch is an invaluable way to test your game and hypotheses before launching globally. It is incredibly important to have well-defined goals and realistic expectations of the data that soft launch can provide.
We would love to hear from you about other soft launch or mobile gaming topics you’d like us to write about in the future. Let us know @JoshLu & @DougMcCracken!
