If you’ve ever ran a product or feature beta before, chances are you’ve experienced some challenges along the way—maybe it was tough finding enough participants, or maybe it was difficult to prioritize what feedback to address before release.
If you’ve never ran a beta before, maybe you’re looking for a general guide on how to do so.
In either case, we wrote this blog to show you how to run a kickass beta test, Influitive style. If you’re wondering what exactly that entails, just imagine a normal beta and then add—you guessed it—advocates.
But why do advocates make for a better beta, you ask?
Well, if you’ve been nurturing advocates through a formal advocate marketing program, you’ll find that their investment in your products and genuine love for your brand makes them eager hand raisers who are invested in giving high-quality feedback.
And, if you’re running your beta through an advocate marketing platform, you’ll find it becomes easier to manage the participants, collect feedback, and reward advocates, leaving them excited to participate again in future betas.
So without further ado, I’m going to share our 5 step guide to running an advocate-powered beta, complete with examples from the beta we ran for our latest major feature release, Campaign Manager.
1. Decide on objectives and how you’ll measure them
Before you do anything at all, it’s important to understand why you’re doing it. Determine if what you’re releasing justifies the time and resources a beta involves. For example, if your new feature requires considerable volume and you’re working under tight deadlines, it may be more beneficial to release and learn along the way.
If you do have the time and resources to run a beta, it’s time to get clear on what you’re hoping to accomplish by setting your goals and objectives. What are you hoping to learn?
You may have end-user interface goals like increasing user adoption, and/or administration goals such as reducing admin burden, or measuring scalability and performance across all customers.
You should also determine what benefits your advocates will get out of the beta. How is this experience going to benefit them? For example, your advocates will get the opportunity to try out new features before anyone else, and they can influence the product so it’ll be most helpful to them in the end.
After you’ve set your goals and objectives, decide how you’re going to measure them. Set concrete and measurable targets to understand what success with your new product/feature looks like, and to provide insight into any pitfalls that may need to be addressed before the final rollout.
How we did it:
We decided to run our Campaign Manager beta for a number of reasons.
We wanted to:
- Learn how well the feature was being adopted.
- Validate that our pre-packaged advocacy campaigns were an effective way to save admin time and effort, while increasing advocate engagement.
- Drive early awareness of the new feature to build excitement before its release.
We also wanted to give our advocates the opportunity to influence and provide feedback on the feature, so we could address any suggestions before general release.
We knew advocates would benefit from trying our pre-templated advocacy campaigns before anyone else, and they would relish the opportunity to shape how the feature would evolve.
We set metrics and targets around feature adoption and performance. Our targets were to see at least 20 of our 35 beta participants use the Campaign Manager tool to publish a campaign, and that admins were actually saving time using the new feature vs. without it.
2. Announce the beta to customers and select participants
Without an advocate community, sourcing beta participants can sometimes feel like pulling teeth. Trust me, I know. Having run betas at previous companies, I know how hard it can be at times to gather enough hand raisers and to get them to engage.
What makes an advocate-powered beta so potent is the nature of the participants. While every beta is run with customers, advocate powered betas are run with your best customers. These super users love your brand and are more invested than the average customer in your company’s success.
If you have been continually engaging your customers and providing value beyond your solution through a customer community, you’ll find that you have eager and willing advocates to call upon when the time comes for sourcing hand raisers.
Instead of pitching the ask, you can present the ask as an opportunity to participate in an exclusive, early-access trial of your new feature. Your advocates should appreciate the opportunity to get their hands on your latest update and will be more than eager to have their voices heard when it comes to providing feedback, especially if you have a proven track record of showing your appreciation for it.
Our Challenge in VIP to welcome Beta participants
Once you’ve built your list of hand raisers, you may find that you actually have more than you need. This is a great problem to have! In this case, thank all for their interest, and select those who most face the problem you are solving with your new release, and who will be able to take the time to provide quality feedback.
How we did it:
We made the initial Campaign Manager announcement in-person, with a bang. We unveiled the feature on the main stage at our last Advocamp conference, and leveraged the energy of the live event to ask for beta participants. While our goal was to source 10-15 participants, we had a whopping 80 hand raisers—we selected 35 and waitlisted the remainder.
Eager campers listening to the product keynote at Advocamp 2017
We then held an AMA-style webinar prior to the beta launch with our participants, so that we could set expectations and educate them about the feature, how to use it, and how to find support during the beta.
We also built excitement internally, by hosting a company wide Lunch & Learn (complete with free sushi!) to answer questions and build awareness of the initiative within the company.
3. Launch your beta
Once participants are selected, and internal and external awareness are built, it’s time to launch! Make sure to monitor and collect data on your metrics from step one. Provide an easy way to educate customers throughout the beta, with resources like manuals, or a knowledge hub.
Make a big deal of the launch—send creative and energetic communications to get your advocate testers excited to provide their thoughts!
While the beta is running, be sure to hold cross-functional weekly standups for product marketing, product management, and development teams to review any reported bugs or customer feedback for the week.
How we did it:
By running our entire beta through our own customer advocacy program (Influitive VIP) we were able to provide beta participants with targeted content that linked to educational tools which could be accessed at any time.
Additionally, we created evergreen feedback Challenges to collect ongoing feedback from participants. Responses to this Challenge triggered an email notification to our team anytime a new response was received.
During the beta, we also acknowledged and rewarded participants who did amazing things with the new feature through Challenges that publicly recognized them to the beta group. This was intended to give star participants some well-deserved props, while inspiring others to get engaged and see what Campaign Manager could do for them.
Publicly recognize your super-participants to encourage engagement
We also used the beta experience to communicate the availability of a complementary product offering which could be purchased to enhance the functionality of Campaign Manager. This helped us to build awareness for a potential upsell in a non-intrusive way, as customers could decide whether they were interested in learning more.
4. Close off the beta and evaluate feedback
Once you’ve collected your mountain of feedback, it’s time to evaluate and prioritize what needs to be addressed before you can release the feature for general availability. If you’ve kept your feedback organized, it should be easy to spot patterns in feedback for common questions, comments, or requests.
If you’ve identified any bugs or recurring requests that need to be addressed, determine how long those will take to address with product and development teams and allow those to determine the final release date. Ensure that you are confident that you’ve adequately solved any issues before going ahead with release.
If you get stuck on what to prioritize (maybe there are several different competing design suggestions or feature requests), list the top options and ask your advocates! They will appreciate the opportunity to help decide.
How we did it:
After collecting ideas and feedback for design and feature requests, we listed them and threw it out to our beta advocates for a vote. We wanted the updates we implemented to be largely driven by our advocates.
After determining the most popular requests, we set forth to finalize a feature that was designed for our advocates, by our advocates. They are the ones who will be using the feature at the end of the day!
5. Reward advocates for their participation
Finally, you’ll want to thank and recognize your advocates for participating in your beta and taking the time to provide you quality data and feedback. If you want to set the stage for the next time you need to call on your advocates, it’s crucial that you follow up to let them know how their feedback impacted the final product.
Take the time to recognize those participants that went above and beyond to help provide quality feedback. Making them feel appreciated and heard goes a long way towards strengthening your relationship with these customers.
You can also ask your participants if they would be interested in being part of any future beta tests. When the time comes to ask again, you can confidently call on the advocates who said “yes.” For those who indicate “no,” you can take the opportunity to gather feedback about their experience participating in the beta, so you can improve the experience for future participants.
How we did it:
We sincerely thanked all beta participants for their time, insights, and immense help. We also gave extra recognition to those who went above and beyond to provide detailed, high quality feedback. We sent them gift cards as a thank-you, plus we shouted them out in front of their peers through our VIP program.
Show appreciation for participants who provided great feedback
Showing advocates your appreciation is a crucial part of closing the feedback loop, and keeping them happy in the long term. With regular nurturing, engagement, and recognition, you’ll cement a strong relationship with advocates—and never have a shortage of eager participants to power your beta tests for years to come.
