Meta-Scientific Research in Marketing: Understanding the Impact of Researcher Choices
Objectives
Highlight the sensitivity of research findings to researcher choices
Provide conclusions regarding better practice in marketing science
Estimate price elasticities of meat substitutes, using large-scale household panel data from 8 countries
Bring together scholars interested in conducting research according to Empirics-First Approach
Please watch our recorded webinar to learn more about this project via this link! Passcode: W+?uD%X9
Please also have a look at our Slide Deck from the Webinar!
About
- How do our choices as researchers affect the results of our research projects?
- What can we learn from the same data being analyzed by different researchers and meta-analyzing these outcomes?
- Contribute and be a co-author of one of the first marketing papers to investigate how researchers impact outcomes.
Progress in the social sciences stems from researcher decisions with respect to a theoretical framework, a set of hypotheses, operationalizations of constructs, sampling, modeling and interpretation – and the acceptance of these decisions by knowledgeable peers (i.e., via the review process) and the research community (e.g., via citations). Some of these decisions come with well-known weaknesses, for example sampling or measurement error, which are being addressed through providing, for example, confidence intervals (e.g., McShane et al. 2024) or triangulation. Another decision in this evidence-generating process has only recently become a focus of attention: choosing a pathway amongst the many analytical options to derive learnings from data (e.g., Huntington-Klein et al. 2021, Sarstedt et al. 2024). Why this specific path is chosen is rarely discussed, and its relative merit is hard to evaluate given the lackof understanding of whether and how taking different forks in the analytical decision-making process would have changed the evidence generated. The resulting potential for heterogeneity in learnings derived from the same set of data (Menkveld et al. 2022) is particularly problematic since its size is unknown, and the reasoning of a (group of) researcher(s) for picking a specific (among many available) analytical path is hardly discussed.
Recently, open science initiatives have called for more efforts to engage in reproducible science that is transparent and helps to improve the quality and accumulation of scientific knowledge (e.g., Deer et al. 2024, Munafò et al. 2017). Such a call should resonate strongly in marketing, a discipline that has been shown to boast an unusually high incidence of p-hacking and publication bias (Brodeur, Cook, and Heyes, 2022). Our call for papers responds to this demand for more transparent research by specifically addressing the issue of heterogeneity in researchers’ path choices and applying an empirics-first (EF) approach (Golder et al. 2023). EF is not guided by hypotheses nor sticking to a pre-committed set of research choices because these choices often arise as a function of the data (e.g., whether a variable has many zeros, is skewed or multicollinear). Nonetheless, researchers need to motivate and document their research choices precisely and comprehensively.
This research project is a collaboration between the International Journal of Research in Marketing and AiMark, a non-profit institution promoting the use of consumer/household/scanner data in enhancing our understanding of marketing and its effectiveness. IJRM has agreed to publish one or more multi-author papers if they pass the dedicated review process for this project.
Schedule
Project Coordinators
Koen Pauwels
Karin Teichmann
Oliver Koll
Hannes Datta
Harald van Heerde
Koen Pauwels
Koen Pauwels is the Associate Dean of Research and Distinguished Professor at Northeastern University. For 4.5 years, he was Principal Research Scientist at Amazon Ads, with brand building and budget allocation recommendations reaching hundreds of thousands of advertisers. Koen received his Ph.D. from UCLA, where he was chosen Top 100 Inspirational Alumnus. After getting tenure at the Tuck School of Business at Dartmouth, he helped build the startup Ozyegin University in Istanbul. Named a worldwide top 2% scientist, Koen published over 100 articles on marketing effectiveness. Koen is editor-in-chief of the International Journal of Research in Marketing.
Karin Teichmann
Karin Teichmann (Dr., WU Vienna) is Assistant Professor of Service Management at the University of Innsbruck. Her research centers around uplifting individuals’ well-being in service interactions with a special focus on open science practices. Karin’s research has appeared in journals like Journal of Interactive Marketing, Psychology & Marketing, Tourism Management, Journal of Travel Research and European Journal of Marketing. Together with her co-authors, she received the Robert Johnston Outstanding Paper Award in 2016 of the Journal of Service Management. Karin is member of the editorial board of the European Management Journal and received the best reviewer award in 2022.
Oliver Koll
Oliver Koll received his PhD from the University of Innsbruck in Austria where he is currently Professor of Marketing. His research focuses on theoretical and practical questions regarding effective brand building, brand/retailer loyalty and stakeholder marketing. His work has been published in leading journals, including the Journal of Retailing and the Journal of the Academy of Marketing Science. Oliver has been affiliated with Europanel (a collaboration between YouGov and Kantar) since 2003 where he supports FMCG manufacturers and retailers in their brand growth strategies. He also is a member of AiMark’s advisory boards in Marketing and Economics.
Hannes Datta
Hannes Datta is an Associate Professor of Marketing at Tilburg University, The Netherlands. His research focuses on developing advanced econometric models to support managerial decision-making and inform public policy in areas such as digital media consumption (e.g., streaming services), branding, and retailing. His work has been published in leading journals, including the Journal of Marketing, Journal of Marketing Research, Marketing Science, and the International Journal of Research in Marketing, where he also serves on the editorial review boards. Hannes is passionate about open science and actively contributes to its dissemination through his involvement at Tilburg Science Hub
Harald van Heerde
Harald van Heerde (Ph.D., University of Groningen) is Research Professor of Marketing at the University of New South Wales. Harald has published widely in the leading marketing journals on marketing effectiveness. He received 10 best paper awards: Journal of Marketing Research (2x), Journal of Marketing (JM; 2x), Marketing Science (2x), and the International Journal of Research in Marketing (4x) plus the Churchill Award for Lifetime Contributions to Marketing Research from the American Marketing Association (AMA). He was elected as an AMA Fellow and received an honorary doctorate from the University of Hamburg. Van Heerde served as a JM Editor.
FAQ
General
- Hannes Datta, Associate Professor of Marketing, Tilburg University
- Oliver Koll, Professor of Marketing, University of Innsbruck & AiMark Advisory Board Member
- Koen Pauwels, Professor of Marketing, Northeastern University & Editor-in-Chief, International Journal of Research in Marketing
- Karin Teichmann, Assistant Professor of Marketing, University of Innsbruck
- Harald van Heerde, Research Professor of Marketing, University of New South Wales & Vice-Chair, AiMark
This project is open to the scientific research community. To participate, at least one member of your research team must have an active affiliation with (or be enrolled as a PhD student at) a university or research institution.
Your primary responsibilities are to provide brand price elasticity estimates for a preselected set of 68 brands in the product category of meat substitutes, including the respective standard errors/confidence bounds. In this project, we are interested in estimating the EFFECT of a 1% brand price increase on brand volume sales, while keeping everything else constant.
We welcome researchers from around the globe and aim to include up to 200 researchers/research teams.
Yes, the University of Innsbruck’s ethics review board reviewed and approved the project (ref: 09/2025).
Data
We use data from household panel providers Kantar Worldpanel and YouGov in collaboration with AiMark, a non-profit organization supporting academic research in marketing and economics. The dataset includes purchases of meat substitute products from 2014 to 2023 across nine countries (some years are unavailable for certain countries).
Yes. If you do, please provide the posterior mean (as an analogue of “estimate”), the standard deviation of the posterior (as an analogue of “standard error”), and the equivalent of a p-value (posterior mass at the other side of 0 as the posterior mean).
Timeline, Tasks and Submission
What is the brand price elasticity (BPE*) for a preselected set of 68 brands in the product category of meat substitutes?
In this project, we are interested in estimating the EFFECT of a 1% brand price increase on brand volume sales, while keeping everything else constant.
The timeline (which can be subject to change) is:
- Early February 2025: the International Journal of Research in Marketing hosts a webinar to promote/explain the project plus Q&A to promote and provide guidelines
- February 25, 2025: deadline to register for research teams
- March 5, 2025: the project team provides links for filling out survey and data access
- June 15: deadline for submission of the findings by research teams
- Second half of 2025: the project team collates the findings and creates a draft paper, where all research team members are co-authors. The objective is to publish this paper in the International Journal of Research in Marketing. The paper will have to pass the review process.
Accepted researchers/teams must:
-
- Sign a non-disclosure agreement (NDA).
- Complete a short survey.
- Perform analyses to calculate price elasticities and standard errors.
- Upload their results.
- Submit their working code (e.g., in R, Python, Stata, etc.).
- Answer a final survey.
You will have to upload a copy of your passport/ID.
After we receive your signed NDA, you will be provided with a link containing (a) a short questionnaire and (b) access to the data repository and its detailed description.
A submission template will be provided within the survey. Use it to submit price elasticities and their standard errors for each chosen brand in all countries.
Please upload your code along with your results when submitting. At a minimum, you must provide the following:
(a) All code used to process the data and generate the final estimates
(b) A detailed “readme.txt” file that includes instructions on how to run your code, the order in which the scripts should be executed, and the required software packages.
(c) A final output file named elasticities.csv that contains the final estimates for each brand and country, including the estimates, standard errors, and p-values.
For further guidance, please refer to Deer et al., 2025, with a particular focus on the section on “documentation” as well as Table 1 and Table 3.
Application process
Please fill out the registration form, which you can find under Register Now.
Your registration will be checked. You will receive an email letting you know whether you can participate in the project.
If your application has been accepted, you will receive an e-mail with a link that will take you to the pre-survey. As soon as you have answered this, you can download the data set and the result template.
Perform analyses to calculate price elasticities and standard errors.
As soon as the analysis phase has expired, you will receive another e-mail that will take you to the post-survey. After you have answered this, you can upload your code.