The 2014 Membership Marketing Survey is Now Open!

It's that time of year again! Marketing General Incorporated's (MGI) Annual Membership Marketing Benchmarking Survey is now open.

This is the sixth year in a row that the survey is being conducted. The goal of the survey is to provide association executives with comparative data on how other organizations recruit, engage, and renew members. Nearly 700 individual member and trade associations participated in the 2013 study. This year, we hope to achieve 1,000 responses. 

We invite you to participate in this year's survey. It should take about 20 minutes to complete. To thank you for your participation, we will send you a FREE prerelease copy of the final report: The 2014 Membership Marketing Becnhmarking Report. 

Click the link below now to get started.

Take the Survey

Please be sure to check back on our blog periodically for initial findings and insights on this year's survey.

Thank you for your time and best of luck to you and your organization in 2014!

Statistical Significance: What exactly does it mean?

What does it mean to be statistically significant?

Significance, as it applies to statistics, is often misinterpreted or misunderstood. Unlike the traditional meaning of significance which implies that something is important, statistical significance suggests that a relationship between variables is not due to random chance or dumb luck. A statistically significant finding may be important or unimportant. However, it simply means that we assume that the relationship between variables actually exists in reality, and is not happening due to chance or error.

For example, let's say that a team of researchers from ABC University release a report from their recent study which suggests that men are significantly more likely than women to suffer from depression. The statistically significant relationship, as observed in this study, implies that the results did not happen due to random chance or error. In other words, this study argues that men are more likely than women to suffer from depression in general, not just within this particular study.

How is statistical significance determined?

Testing for statistical significance begins with a null hypothesis (i.e., we assume there is no relationship between the variables being tested). Researchers then use statistical tools to determine a p-value or the percent likelihood that a result happened by chance. Typically, the standard cut-off point for a p-value in social science research is .05. In other words, p-values lower than .05 (.04, .01, .001) would imply that the relationship between variables is statistically significant. A p-value of .05 assumes that there is a 5% chance that the relationship between variables is due to chance. A p-value of .01 would imply that there is a 1% chance that the relationship occurred by chance. In either of these cases, we would reject the null hypothesis and conclude that there is a statistically significant relationship between the variables. Furthermore, we believe this relationship is not being caused by chance or error.

P-values larger than .05 (.051, .06, .1) would not be considered statistically significant since the chances that the results happened by chance is larger. For instance, a p-value of .1 assumes that there is a 10% chance that the relationship between variables is happening by chance. In this case we would accept the null hypothesis and conclude that the relationship between the variables being tested is not statistically significant.

Let's use another example. Imagine that we conduct a study to find out if men or women have higher IQ scores. Our null hypothesis would be to assume that there is no relationship between gender and IQ scores. In other words, we expect that there are no differences between men and women and their overall IQ scores. Upon analyzing our data, however, we discover that the average IQ for men is 100, and the average for women is 106. Since our results suggest that women are more likely to have a higher IQ score, we must rely on our p-value to tell us whether or not these results happened by chance or luck.

After doing our statistical calculation, we find that our p-value is equal to .061. Since .061 is above our threshold of .05, we conclude that the relationship between men and women and their IQ scores is not statistically significant. Even though women scored higher than men, our p-value tells us that the difference between scores may be the result of chance or luck. In this case, we would accept the null hypothesis and say that there is no relationship between gender and IQ scores. 

How should significance be interpreted?

Just as correlation does not imply causation, statistical significance does not mean that the relationship we are observing will happen 100% of the time. When interpreting significance, it's important for the reader to examine other aspects of the research such as how large or how small the sample is, the type of sample collected (i.e., Is it a random sample or something else?), and how the questions are asked or presented. Just because a study yields significant results does not guarantee that the research was designed and executed carefully. There's plenty of research out there that can be misleading or interpreted incorrectly.

Qualitative Research vs. Quantitative Research: What is the difference?

Research can be a great tool for membership organizations. When the proper questions are asked, the answers can be revealing and sometimes surprising. Findings can be powerful tools for needed change.

Researchers gathering information about associations and other membership organizations often use two methods of research to obtain one set of results: qualitative and quantitative methodologies. Qualitative research tends to be more exploratory, whereas quantitative research tends to be more confirmatory.

Qualitative research is typically completed first. Researchers ask questions of a small sample of participants and collect data from open-ended questions to uncover themes and patterns. Qualitative inquiry seeks to understand human behavior and the reasons why and how the behavior occurs.

Quantitative research is designed to establish whether patterns and themes seen in the qualitative research are also visible in the larger membership population. Using the qualitative data to fine tune the questions, the quantitative research surveys a large sample to test hypotheses generated from the qualitative research. The overall goal of quantitative research is to have the ability to estimate the thoughts, attitudes, and behaviors of a specific population.

How does the data collection from each methodology differ?

Associations often use these research methods to gauge member perceptions about the organization and to find out whether members are pleased or disappointed by the benefits and services a group offers its members.

Qualitative information is often gathered in focus groups, telephone interviews, research intercepts, and more recently using online bulletin boards and discussion groups.

Quantitative information is usually collected through written or online surveys. Using an online methodology rather than a written one, is far less expensive, faster, and because of the widespread use of computers in all age ranges, it is no longer considered a biased medium for data collection.

How is the information from each methodology used?

Both types of data are valuable sources of information. They are used in somewhat different manners in order to achieve an end result that is truly informative.

Qualitative research relies on the use of open ended questions, seeking opinions as well as factual information. Questions are designed to be interesting, stimulating, but not leading. Because the information is often gathered from a small sample, the data is used as a directional guide to help refine and focus larger, quantitative studies.

Quantitative research consists mostly of closed-ended questions (those with answer choices provided for the respondent). The research generally includes many participants designed to discover the prevalence of agreement or disagreement with the ideas and opinions revealed in the qualitative research. With a large enough sample, the findings are considered representative of the population and can be used as the foundation for strategic decisions and strategies. (See our previous blog post on sample size for questions regarding what is representative of your population.)

Both qualitative and quantitative research questions used in member research often ask about reasons for joining, programs that are most important and why, professional challenges being faced, resources that are desired but not offered, and perceptions about the association’s brand and tagline. Demographic information is also collected in order to understand differences between segments within the membership.

While the two research methods are different, each has a significant role in the process of inquiry and discovery. Better understanding of the two can give you greater control and utility the next time you conduct research. When used together properly, these two approaches can yield actionable data your association can use to better understand membership.

Determining Sample Size: How Much is Enough?

Many associations who are about to implement a large quantitative survey where the size of the population is unknown (e.g., surveying the total number of prospects within a market) often ask us the question, "How large of a sample do we need to get good results?" The answer to this is that there is no finite number that determines whether a sample is "good" or "bad". Instead, there are other factors that need to be taken into consideration. Below are some tips that can help your association determine the appropriate sample size to meet its specific needs when dealing with a population size that is unknown.

• Decide how much error you want associated with the results: Since it's nearly impossible to survey every single member of larger populations, statisticians use a formula to determine the error associated with results. This statistic is known as the margin of error or the assumed error that corresponds to a specific sample size. The chart below summarizes how the margin of error decreases as more responses are obtained.

Margin of Error	Number of Responses Needed
+/-9.8%	100
+/-6.2%	250
+/-4.4%	500
+/-3.6%	750
+/-3.1%	1,000
+/-2.8%	1,250
+/-2.5%	1,500

All of these percentages are calculated at a 95% confidence level, meaning that if we were to conduct the same survey 100 times, we would get similar results, plus or minus the margin of error, 95 out of 100 times.

• Decide what your budget will be: Not only is it nearly impossible to survey everyone in a large population, but it can also be quite costly. The beauty of statistics is that you don't need to survey everyone to get results that will provide you with direction. Prior to conducting your survey, determine how much money you have in your budget to perform the research. This will give you a better understanding of the sample size you can afford to get.

• Decide what your timeline will be: Obtaining large sample sizes can take a significant amount of time. Depending on what your needs are, allow enough time for data collection so you are able to get enough responses that will give you a margin of error that satisfies your needs for the research.

Questions? Comments?

Contact us:

Erik Schonher, MBA

Phone: 703-706-0358

Email: Erik@MarketingGeneral.com

Dr. Adina Wasserman, PhD

Phone: 703-706-0373

Email: AWasserman@MarketingGeneral.com

Jeff Tranguch, MA

Phone: 703-706-0364

Email: JTranguch@MarketingGeneral.com

Market Penetration and New Member Retention

In marketing it is often said that acquiring new members is more expensive than retaining current ones. That doesn't mean effort shouldn't be made to acquire more members, but current research now shows that retention of new members is positively correlated with overall market penetration (see chart below).

Market Penetration by New Member Renewal Rate
Market Penetration	New Member Renewal Rate
	Less than 60% Renewals	60-79% Renewals	80% Renewals and Higher
Less than 60%	85%	68%	52%
60-79%	7%	17%	20%
80% and higher	8%	16%	28%

This seems like an obvious statement, right? The more members retained, the greater the overall market penetration. However, the chart below illustrates that this correlation is not present when compared to the change in overall renewal rates (new members and current members).

Market Penetration by Overall Change in Renewals in Past Year
Market Penetration	Overall Change in Renewals in Past Year
	Increased	Unchanged	Decreased
Less than 60%	70%	67%	73%
60-79%	17%	14%	15%
80% and higher	13%	19%	13%

Why not? If an association shows an increase in overall renewals, shouldn't the market penetration improve as well?  One theory is that new members not only revitalize an association, but are more inclined to share their positive experiences, and our research demonstrates that word-of-mouth recommendations are the number one method for becoming aware of an association.

As only 37% of participating associations report first year member renewals at 80% or higher, onboarding processes and member engagement programs become increasingly important. Evaluate the onboarding mechanisms employed by your association, and measure the awareness and usage of engagement programs by first year members. This group is more important to overall market penetration than realized. 

Data taken from Marketing General Incorporated’s 2013 Membership Marketing Benchmarking Report. 

Using Research to Grow Membership

Understanding the needs and wants of membership is vital to an association's growth. Whether it's qualitative or quantitative, research provides an association with a "blueprint" of members' thoughts and opinions toward a specific set of ideas and concepts. Utilizing this blueprint, the association can tweak its products, benefits, and services to better meet the needs and wants of membership and, in turn, maximize growth and revenue.

To support this argument, we have included data below taken from the 2013 Membership Marketing Benchmarking Report. The following charts provide a summary of growth statistics for associations who introduced member research in 2012.

GROWTH STATISTICS FOR INDIVIDUAL MEMBERSHIP ASSOCIATIONS WHO INTRODUCED MEMBER RESEARCH IN 2012
	Change in Membership	Change in Renewal Rate	Change in New Members
Increased	59%	45%	65%
Stayed the Same	16%	32%	20%
Decreased	26%	23%	16%

GROWTH STATISTICS FOR TRADE ASSOCIATIONS WHO INTRODUCED MEMBER RESEARCH IN 2012
	Change in Membership	Change in Renewal Rate	Change in New Members
Increased	53%	38%	72%
Stayed the Same	21%	28%	16%
Decreased	26%	34%	13%

The findings above demonstrate that conducting member research can increase the likelihood for an association to experience growth in overall membership, renewals, and new members. Although it is not directly related to growth, research will provide the association with a clearer picture of where it is doing well and where attention needs to be focused. With this knowledge, the association can take the necessary steps to improving membership and ultimately, increase the potential for membership growth and profitability.