R, the FDA, and clinical trials

There’s been an interesting discussion recently on the R Project group on LinkedIn (sub. req.) on the topic of using R for clinical trial analysis in the pharmaceutical industry. The questions asked were:

Is the number of [FDA] submissions featuring R growing? Are companies happy to use R for submission work yet? Is the use of R within regulatory agencies growing? If not, how do we move the discussion forward both internally within companies AND externally with regulators?

This is a topic we’ve covered here before, but Marc Schwartz from MedNet Solutions posted such an excellent reply to that thread that I repost it here in its entirety (with his permission):

The exact answer is that there is nothing, including 21CFR11, preventing you from using R for a submission. There are those who believe that 21CFR11 does not apply to R or any other application, including SAS, used purely for statistical analysis and presentation. 21CFR11 is intended to be used in the domain of source electronic records and their management. If R, or SAS for that matter, are not used for that purpose (eg. you use Oracle for data acquisition and management), then there is the basis to believe that 21CFR11 is not …

There’s been an interesting discussion recently on the R Project group on LinkedIn (sub. req.) on the topic of using R for clinical trial analysis in the pharmaceutical industry. The questions asked were:

Is the number of [FDA] submissions featuring R growing? Are companies happy to use R for submission work yet? Is the use of R within regulatory agencies growing? If not, how do we move the discussion forward both internally within companies AND externally with regulators?

This is a topic we’ve covered here before, but Marc Schwartz from MedNet Solutions posted such an excellent reply to that thread that I repost it here in its entirety (with his permission):

The exact answer is that there is nothing, including 21CFR11, preventing you from using R for a submission. There are those who believe that 21CFR11 does not apply to R or any other application, including SAS, used purely for statistical analysis and presentation. 21CFR11 is intended to be used in the domain of source electronic records and their management. If R, or SAS for that matter, are not used for that purpose (eg. you use Oracle for data acquisition and management), then there is the basis to believe that 21CFR11 is not relevant.

That being said, there are other FDA documents that would still apply, including GxP and the others relative to the validation of software, which are listed in the “R-FDA” document that has been referred to.

This means that you (not R Core nor the R Foundation) still have an obligation to implement and document SOPs and related IQ/OQ/PQ processes relative to R. How you do this is up to you and your organization based upon existing SOPs and any relevant internal legal and regulatory guidance.

The “R-FDA” document provides the basis to express the SDLC for R (Base and Recommended packages). There is also documentation provided as to how R meets the relevant 21CFR11 components, so that those folks who are not convinced that 21CFR11 is not relevant, can understand how R fits within that framework.

With respect to contributed packages, the obligation for those is no different than if you were to use your own code. Provide documentation and SOPs for IQ/OQ/PQ based upon your own processes. The key is that you can document how you have established that the results from R and any other contributed code are consistent and reliable. See section 5.2 “Validation” in the R-FDA document.

The question keeps coming up because of FUD, perpetuated by those who are loathe to open their minds or by those with a vested fiduciary interest in seeing other applications used. Bear in mind, there are those who still believe that SAS is the only application that can be used for FDA submissions. The consulting companies whose business model is dependent upon making MS Access and Excel 21CFR11 compliant and validated would have an argument there… 🙂

The FDA will not come out formally with a position that endorses any application, so you will not get an answer from them formally stating anything. They are precluded from doing that as a government agency.

That the FDA is using R internally and for rather high profile cases (eg. Avandia) should be cause for increasing one’s comfort level.

This does not mean that the challenge for R is not present. Risk averse internal management and decision makers have to be sold on the use of R and that it is free may not be sufficient, given the costs of converting legacy code, processes and training. Even in a time of cost containment, the potential savings of not paying significant sums for annual SAS licenses may not be sufficient motivation.

If the organization is already using open source software internally (eg. Linux, Apache, SVN, Firefox, Thunderbird, OpenOffice) then there is at least an open mind to the potential use of R. If not, then the challenge may be more profound.

If R is already being used for non-clinical applications, which is presently the case even in large pharma, then there is the potential opportunity to educate the clinical statisticians to improve their comfort level. This would provide a bottom up strategy for R, if you can gain the comfort and support of the folks doing the work. That can help to make the argument to senior management somewhat easier. However, at the senior management level, you need to be able to make a solid business case for the change.

This situation is not going to shift overnight. It is part and parcel of the fundamental resistance to change by all of us. It will take time and people willing to step up and do the work within their own organizations to change behavior and overcome the resistance. 

I think Marc’s response is spot-on. The key is that the burden falls on the organization using R to verify GxP and 21CFR11 compliance, insofar as they may apply to the specific context in which R is being used for clinical trial analysis. Fortunately, this is relatively straightforward with R. REvolution Computing helps companies with this process of validating their use of R in context, and by providing a distribution of R (REvolution R Enterprise) with a documented build process ready for audit.