Stat Sig Ain't The Whole Story
We have all heard of A/B tests and likely have attempted to do one ourselves. For those unfamiliar, an A/B test is one in which a variable is changed and cell A is compared via some type of statistically test to cell B. But have you heard of A/A tests? It's exactly as it sounds. You test the exact same thing against itself. Why would you ever do such a thing? Because the majority of advertisers are often ill-equipped to run a proper A/B test, and the A/A test is a concept that will help verify the validity of your testing procedure.
The author of the article, A/A Testing: How To Increase Conversions 300% by Doing Absolutely Nothing, showcases a fantastic use case of A/A testing with his email distribution list. Even with an email list of 30,000 the author finds many aspects in his A/A test that show a statistically significant result... but it was the exact same thing tested against itself.
"To many 'wantrepreneurs' (my former self included), this looks like 'oh wow, you increased opens by 10%!' They may even punch it into Visual Website Optimizer’s significance calculator and see that p=.048. 'It’s statistically significant!' they (or I) might exclaim.
…to a trained statistician, there is nothing remarkable about these 'results'. Given the baseline conversion rate on opens, the sample size simply isn’t large enough to get a reliable result. What’s happening here is just the silly tricks our feeble human minds play on us when we try to measure things."
While this may sound like it doesn't apply to your tests, this is the exact thing happening with many A/B tests. Results that are interpreted as gospel because it indicated something incremental, and thus a new tactic is chosen. But what was your confidence interval? What was the sample size or power of the test (a concept we will deep dive into on a future newsletter)?
"Even if you do have a large enough sample size, you’re bound to get the occasional 'false positive' or 'false negative.' Meaning that you could make the completely wrong decision based upon false information."
We're not saying A/B testing is bad or wrong, quite the opposite. It's an incredibly powerful tool that can provide directional information, but as with many tools it can become dangerous when used incorrectly or when managed by people who don't adequately understand the variables at play.
Before your next A/B test, ask yourself a handful of questions to decide whether you should be running:
- Do I have an important question? Will answering this question make an impact worth the effort of running a test?
- What else could I be doing with my energy and money? By the nature of testing, it's an overall less efficient use of funds than scaling a "winning" strategy.
- Can I run a large enough test? Use this sample size calculator to see if you're actually able to reach statistical significance.
- Do I really understand testing? Try running an A/A test to get a feel for how misleading “results” can be. Be honest with yourself around your ability to interpret results, because the final decision of A versus B with potentially thousands or millions in ad budget (and thus revenue for the business).
If you want to double-click into this topic, we recommend reading "Why Data, Statistics and Numbers Can Make You do the Wrong Thing". Lengthy but extremely informative on the pitfalls of A/B testing.
--
This write up is part 4 in the series we call Adventures in Significance. After running 257 lift studies across 30+ different clients in our past, we thought it would be helpful to write a series on common mistakes and how to avoid them.
This is a dreaded subject by many but is core to the direct response advertising world. We thought we could do our part with a round-up of our favorite opinions on the subject and how this applies to your business.
--
Thanks to Science in HD for sharing their work on Unsplash
Read next
Facebook Ads Bidding Strategies can either make or break your advertising campaign. If you've been struggling with getting the best results, understanding the benefits and drawbacks of each strategy can save you both time and money. The right bidding strategy can help you reach your target audience more effectively and get the most out of your advertising budget.
Each bidding strategy has its unique benefits and challenges. Some are great for maximizing visibility, while others prioritize cost-efficiency. Choosing the right one depends on your specific goals, whether that's more clicks, better engagement, or higher sales. Knowing the pros and cons of each strategy will help you make informed choices that benefit your business.
This article will guide you through five key Facebook Ads Bidding Strategies. You’ll learn about their benefits, drawbacks, and how to pick the one that suits your campaign objectives. By the end, you’ll have a clear understanding of which strategy will help you achieve your advertising goals effectively.
Understanding Facebook Bidding Mechanics
Facebook bidding is essential for advertising success. It involves auctions where advertisers compete for ad placements. Understanding key elements like Auction Dynamics and Different Bidding Strategies is crucial.
Auction Dynamics and How Bids Work
In Facebook's auction, ads compete based on bids, estimated action rates, and ad quality. Bid represents how much you're willing to pay for a specific action (like clicks, views, or conversions). The Cost Per Result adjusts based on competition.
Bid Cap lets advertisers set a maximum bid. This ensures spending control but may limit campaign reach. Meta bidding strategies, like Lowest Cost and Target Cost, help optimize for specific goals, balancing cost and performance.
Factors influencing the auction include:
- Bid amount
- Ad relevance
- Estimated action rates
Exploring Different Bidding Strategies
Advertisers can choose from several Facebook bidding strategies. The Lowest Cost strategy aims to get the most results for the lowest price but may lack spending control. The Cost Cap strategy helps maintain an average cost while driving results.
The Bid Cap strategy is useful for high-control needs, letting you set the max bid per action but it might restrict delivery. Target Cost aims for a stable cost per action, ideal for steady budget planning.
Choosing the right strategy depends on your campaign goals, budget, and desired Cost Per Result. Evaluate each option to find the best fit for your needs.
Implementing Bidding Strategies for Campaign Success
Successful implementation of bidding strategies can drive better results and optimize ad spend. Key factors include setting appropriate bid caps, maximizing returns using ROAS goals, and balancing volume and value.
Setting the Right Bid Cap for Your Campaign
Setting the right bid cap involves determining the maximum amount you are willing to pay for a result. This ensures costs don't exceed the budget. Bid caps can help control spending and improve efficiency.
- Analyze past performance: Review historical data to identify the highest bid that achieved desired results.
- Adjust as needed: Be flexible to change bid caps based on real-time campaign performance.
- Consider the competition: Higher bid caps might be necessary in competitive markets.
Maximizing Returns with ROAS Goals
Use the Return on Ad Spend (ROAS) bid strategy to drive maximum returns. ROAS goals ensure that every dollar spent on ads generates a specific amount of revenue.
- Calculate target ROAS: Set a realistic ROAS based on past campaigns.
- Monitor and tweak: Regularly check ad performance and adjust your ROAS goals to meet revenue targets.
- Balance quality and cost: High ROAS might limit reach, so find a balance between cost and quality.
Balancing Volume and Value in Bidding
Balancing volume and value helps achieve the right mix of reach and profitability. Consider using both Highest Volume and Highest Value strategies.
- Highest Volume: Bids are set to get the most conversions, good for awareness and large-scale campaigns.
- Highest Value: Focuses on getting the highest-value conversions, suitable for targeting high-value customers.
By carefully implementing these strategies, advertisers can meet their campaign goals effectively.
5 Facebook Bidding Strategies: Benefits, Drawbacks, and Selection Guide
Static ads and dynamic ads serve different purposes in the world of marketing. Static ads are simple and stay the same at all times. They are easy to create and can be effective for straightforward messaging. But dynamic ads offer customization, changing their content to fit the audience's preferences and behaviors.
Dynamic ads might seem complicated, but they bring better results by targeting specific groups with personalized messages. This means higher engagement rates and more conversions. Static ads, on the other hand, are less effort to produce but may not capture attention as effectively.
Deciding between static and dynamic ads depends on the brand's goals and resources. Each has its strengths and can be powerful if used appropriately in a marketing strategy.
Understanding Static and Dynamic Ads
Static ads and dynamic ads serve different purposes in digital marketing. Each has unique features and benefits that cater to varied marketing needs.
Exploring Static Image Ads
Static image ads are straightforward. They are typically still images that do not change once created. These ads are ideal for conveying a clear, unchanging message or brand image.
A static image can include text, graphics, and logos, and is often used on websites and social media platforms.
Advantages of Static Images
- Consistency: The message remains the same, which can be useful for brand recognition.
- Simplicity: They are simple to create and often cost less than dynamic ads.
- Predictability: Once the ad goes live, what you see is what you get.
Unpacking Dynamic Advertising
Dynamic ads are more complex. They can change content in real-time based on user data and behavior. Unlike static ads, dynamic ads can alter images, text, and calls to action depending on who is viewing the ad.
Benefits of Dynamic Ads
- Personalization: Content can be tailored to each user, potentially increasing engagement.
- Flexibility: They can show different messages to different audiences without creating multiple ads.
- Efficiency: They adapt to user preferences, making the ad experience more relevant.
Comparative Analysis and Use Cases
Static and dynamic ads offer different benefits and limitations. This comparison will help you understand where and how to use each type effectively in your marketing strategy.
Static Images Vs. Videos
Static images are simple and quick to create. They load faster than videos, which is great for mobile users and slow internet connections. They allow for clear, focused messages without distractions.
Videos, on the other hand, capture attention better with motion and sound. They convey more information in a short time. Videos are more engaging and can demonstrate products or services in action.
Leveraging Opportunities for Static Ads
Static ads are useful in various scenarios. Billboards are a great example, as they need to be read quickly. Print ads in magazines and newspapers also benefit from static images. Online banners are often more effective when static, as they load quickly and are less intrusive.
Static ads are best when the message is straightforward. They work well for short calls to action like "Buy Now" or "Sign Up." Visually, they should be clean and uncluttered to convey the message quickly.
