Meta-analysis of data from GE maize fails reality check

Pellegrino et al. (2018) meta-analysis substantially flawed due to bias in data selection and methodological insufficiencies

20 March 2018 / In their meta-analysis, Pellegrino et al. (2018) claim to have found evidence that GE maize performs better than conventional maize in regard to higher yield and lower concentrations of mycotoxins. Further, it is claimed that non-target organisms are not affected by the toxins produced in the GE maize. These findings appeared in the media, but closer analysis shows that the publication is substantially flawed due to bias in data selection.

(1) Relevant issues were left out of the meta-analysis: Unlike studies performed on chemicals or pharmaceuticals, there are very few clearly defined standards in the area of risk research performed with genetically engineered crops. In many cases, the experiments are very specific and lack harmonised standards for study designs.

Clearly defined protocols are an important prerequisite for performing meta-analysis of the data. Therefore, the question is raised to which extent publications on these issues can be subjected to meta-analysis at all. In this case, nearly 90 percent of all relevant studies were not included in the analysis. However, this does not mean these studies cannot generate relevant findings. Rather, if only 76 studies out of more than 6000 studies were taken into account, it is likely that relevant findings escaped this publication simply due to an inadequate parameter for paper selection / exclusion.

Indeed, this problem is evident in the regard to the effects on non-target organisms (NTO): Due to the parameter used for the selection procedure, highly relevant groups of NTOs, such as pollinators and lepidoptera, are not mentioned. Consequently, species-dependent variability within the large families that is decisive for risk assessment is completely left aside. Furthermore, for important groups of organisms, such as larvae from Chrysopidae and Coccinellidae, only a very low number of studies were taken into account. Thus, no general conclusion regarding effects on non-target organisms, such as protected butterflies, can be derived from this study.

(2) Biased selection of the publications: Studies were only included if the isogenic comparator was grown in parallel. This kind of field study design is mostly used in studies performed by industry, conducted for market authorisation. This design is important if genetically engineered plants are to be compared in regard to their composition or their phenotype to the isogenic conventional variety.

However, such studies do not reflect the practical conditions of large-scale maize cultivation. This is relevant for the load of mycotoxins in the harvest: Under practical conditions, farmers will try to take measures to avoid damage if they are aware that pest insects might attack their fields where conventional maize is grown. These measures will have an impact on yield and the mycotoxins load. However, this is not necessarily the case with field trials conducted by industry. In these field trials it is in the interests of industry to show good performance of its Bt plants, but not to reduce damage and load of mycotoxins in the conventionally-grown plants.

Indeed, looking at the 76 publications used in this meta-analysis, it can be shown that industry was involved in more than half of the studies (40). In an additional number of 9 studies, no information on funding is given. As explained above, it is plausible that the involvement of industry can impact the outcome of the studies. Thus, to avoid bias in the underlying data, it would have been necessary to identify the interests and purpose behind each of the 76 studies. In the absence of such analysis, no conclusions can be drawn from this meta-analysis.

(3) Number of studies is too low: The authors have tried to come up with results that allow statements on all kinds of Bt events grown between 1996 and 2016. However, within this period of time dozens of events were allowed on to the market. Currently, in the EU, more than 30 GE maize events are allowed for import. According to data from industry, 100 GE maize events are grown worldwide. Most of them produce Bt toxins. Currently, around a dozen Bt toxins are produced in genetically engineered plants, all belonging to different classifications.

Each of these toxins produced in the plants is different in size and structure compared to the original toxins produced by soil bacteria. Further, each of the events have a specific version and / or a specific combination of these Bt-toxins. So how can an overall number of 76 studies allow a meta-analysis of around 100 events producing many different Bt toxins in various combinations?

In this context it is interesting to note that more than half of the studies used for the meta-analysis were published more than ten years ago. There are only two from 2015 and one from 2016. Thus, in regard to the impacts of the stacked events currently being grown on a large scale, such as genetically engineered maize SmartStax that produces six insecticidal toxins, no reliable conclusions can be expected from this meta-analysis.

(4) No indication of the limitations of the study: A meta-analysis dealing with such complex matters as the impact on the environment and human health in combination will always have some limitations. For example, a list of potential hazards would have been necessary to allow a general statement on the safety of these plants, as would a list of investigations performed per topic. However, no such overview of relevant topics and questions was provided.

In general, the publication clearly fails to identify its limitations and relevant uncertainties. Looking at the claims made by the authors and the substance of their analysis, this publication should have been completely revised before publication.

 

The study:
Pellegrino, E., Bedini, S., Nuti, M., Ercoli, L. (2018) Impact of genetically engineered maize on agronomic, environmental and toxicological traits: a meta-analysis of 21 years of field data. Scientific Reports, 8: 3113. https://www.nature.com/articles/s41598-018-21284-2