With GM crops having been produced for over 25 years now, there is an increasing amount of evidence regarding the degree and diversity of benefits from their production. Typical benefits include reduced chemical use, increased yield and higher farm profitability. Empirical evidence now exists at the farm level for virtually every GM crop commercialized at a global level, with many articles providing detailed evidence for country-specific and crop-specific adoption. Most of this literature further quantifies that GM crops contribute to mitigating climate change. The question that we asked is, how robust is this literature?
To answer this question, we undertook a literature search in 2022 which produced over 70 articles. These articles were analyzed for levels of contribution to mitigating climate change. A policy assessment methodology was applied, allowing for observations to be made about the robustness of the relationship between GM crop production and climate change mitigation.
The Literature
Our literature search found articles across 6 different areas of importance to climate change mitigation. The literature search columns (area, change in chemical use/toxicity, carbon sequestration, greenhouse gases, land use change and soil health) are adapted from the FAO’s major principles for sustainability. The articles that were identified were coded by our five research domains and cross encoded for the key mechanism they were assessing: agronomy, data, genetics and input use, farm structure/size, machinery and policy (Table 1).
- The carbon sequestration literature establishes that the genetics provided by herbicide tolerant (HT) crops allows for changes in agronomic practices, predominantly the removal of tillage as a form of week control. Data is limited, but sufficient at the GM crop adopting country level to estimate national and global carbon sequestration volumes.
- There is no evidence that changes in farm structure (area) play a significant role. This is not to say that farm structure does not play a role, just that there is no literature to date quantifying this relationship.
- Similarly, while there have been significant changes in seeding equipment over the past 30-40 years that have contributed to the ability of farmers to be better able to direct seed into the stubble of the previous crop, we found no literature quantifying any relationship to increased carbon sequestration.
- Studies show that the genetics of HT and insect resistant crop innovations result in significant agronomic impacts.
- In comparisons with the production of commodities prior to the commercialization of GM varieties or in the production of non-GM varieties, evidence confirms GM crops reduce chemical applications. In many developing countries where small landholders have adopted GM crops, these applications will predominantly be by hand, resulting in pesticide poisoning reductions. In industrial adopting countries, applications are done by mechanized sprayers, with chemical use reductions contributing to reductions in GHG emissions.
We found no literature correlating changes in farm machinery with changes in chemical use.
Literature Analysis
Citations and journal rankings are often used to tell us about the quality of publications, however, they are only partial indicators of quality. High citation can be both a signal of good and bad methods while much of the evidence vital to decision making is often lightly cited. There is increasing interest in other measures of quality of evaluations. The Maryland Scientific Methods (MSM) Scale was developed in the 1970s to communicate to scholars, policymakers and practitioners in the simplest possible way the methodological quality of various evaluative studies, by coding them by their methodological quality. Using the approach developed by Ratcliffe (2019) offers a fine-grained hierarchy for evidence ranging from ad hoc confirmatory efforts (level zero) through to well-structured systematic reviews and, at their apex, involve meta analyses of a set of repeated randomized controlled experiments.
The MSM Scale offers seven levels of evidence including:
- Level 0: Those reviews that are confirmatory and have limited value by themselves. Anecdotes, case studies and general reports that are designed to justify effort often have little probative value if they are the only evidence.
- Level 1: Two types of review fit here: cross-sectional ex post comparison of treated groups with untreated groups or a before-and-after comparison of a treated group, without any comparison with an untreated group or any use of control variables. These types of analysis can identify correlations but often are not able to assign causality, partly because without any formally controlled counterfactual, there is no way to definitively show that the measure caused any observed results.
- Level 2: Studies at this level use control variables to do either cross sectional or before and after comparisons, albeit still without any untreated comparison groups. These types of studies may be able to establish a causal order but fail to rule out many threats to internal validity, in that there may be other explanations for what was observed.
- Level 3: The minimum standard for substantive evaluation is that the measure be structured as an experiment, with a well-defined source population that fits with the policy area, and where there are both treated and non-treated subjects that can be assessed for behavioral responses. In effect these could be considered a quasi-experiment or randomized control trial (RTCs). Both causality and scale and scope of impact of the measure can be discerned in such studies.
- Level 4: This level of evidence draws on the repeated use of RCTs to control for other variables. This helps to remove the chance of spurious correlations from a single assessment of any venture.
- Level 5: Where Level 3 and 4 evidence is drawn from purposeful construction of in-experiment subjects and controls, Level 5 studies remove that restriction. Level 5 studies apply the same experimental methods to randomly assigned populations, so that the causal assumptions can be validated more generally.
- Level 5*: Once a body of evidence through a range of Level 3 and 4 studies has accumulated, sometimes there is value in doing a study of studies, to discern the meta results and sensitivity analysis of the influence of different modeling approaches and assumptions on the derived outcomes.
We used this rubric to code all the literature, undertaking the following steps:
- In order to have a proper ordering, we rebased this MSM scale to 0 = 1, 1 = 2, and so on to 5* = 7.
- We first coded every article by its impact: area, chemicals, carbon sequestration, GHG, land use and soil health.
- We entered in their citation rate as of September 13, 2022.
- We coded every article for where it would fit in the MSM scale, from 1-7.
- We then ran reports to understand the nature and quality of the work being done.
Our assessment illustrates where this literature is still emerging (Table 2). We found 74 articles that offered structured evidence of environmental impacts. In a few cases the articles looked at more than one mechanism (agronomy, data, genetics, farm structure, machinery or policy) and a few explored more than one impact (area, carbon sequestration, chemicals, GHG, land use, soil health). Those articles that are multifactorial are included more than once.
One can see that some of the oldest work was really case studies and anecdotal evidence (MSM 1) and that that work has a relatively low citation rate. Investigators have taken on more advanced methods (MSM 2-4) as the technology has matured and that body of work is getting taken up and cited. More advanced RCTs with counterfactuals (MSM 5) is just getting going. We found only five lightly cited articles that did level five RCTs and they were on average only seven years old. We could not find any that have run these studies long enough to qualify for MSM 6, which requires repeated RCTs to remove sampling bias. We found a range of meta analyses, but one must keep in mind these surveys can only be as good as the underlying literature, which still lacks full rigorous RCTs.
Different impacts have been differentially studied. So far chemical, soil health and GHG effects are well articulated, albeit without repeated, full randomized control trials with confounding variables (which would code MSM = 6). Nevertheless, we found more than 10 articles in each area, and at least one meta-study in each domain. There were fewer articles about carbon sequestration but we found they used a somewhat more rigorous methodology and offered at least one meta-study (MSM = 7). Relatively speaking the literature on area and land use is a bit weaker in terms of method, without any full randomized controls or meta studies. On average, the literature is 11 years old, with the oldest dating back to 2000. Apart from the impact on area, most of the literature is distributed around the mean of 11 years.
Observations
While the challenges facing agriculture are numerous, two key ones stand out: mitigating climate changes and contributing to the United Nation’s (UN) Sustainable Development Goals (SDGs), especially the first three of reduced poverty, less hunger and better health. Innovation will be the cornerstone of ensuring that agriculture and food production is able to be done such that yields and nutrition are maintained through climactic change. Discussions at the Convention on Biological Diversity’s Conference of Parties Meeting on Biodiversity (COP-15) held in Montreal in December 2022, highlighted the visceral opposition of some to protecting biodiversity by such beneficial and safe technologies as the use of genetic modification technologies. Rejecting empirical evidence of the benefits from GM crops ensures that these contributions to mitigating climate change and achieving the SDGs will be dramatically minimized in the future.
A growing volume of research contributes to quantifying the environmental benefits from GM crops, which helps regulators, scientists and risk assessment experts make decisions that advance plant technologies that contribute to mitigating climate change and achieving the SDGs. In the absence of these innovative technologies, countries will find it increasingly challenging to meet both their climate mitigation and SDGs objectives. Scientifically indefensible regulations are a large obstacle to safe technological solutions for many of the myriad problems and challenges facing agriculture and climate change mitigation. Better and more evidence will be necessary to move forward.
Interested in reading the full article this blog is based on? Please follow this link: An Assessment of the Linkages Between GM Crop Biotechnology and Climate Change Mitigation (Smyth et al. 2024)
