Proteomics Viewpoint Articles
Comments to the Viewpoint article
Clinical proteomics: A need to define the field and to begin to set adequate standards
Harald Mischak, Rolf Apweiler, Rosamonde E. Banks, Mark Conaway, Joshua Coon, Anna Dominiczak, Jochen H. H. Ehrich, Danilo Fliser, Mark Girolami, Henning Hermjakob, Denis Hochstrasser, Joachim Jankowski, Bruce A. Julian, Walter Kolch, Ziad A. Massy, Christian Neusuess, Jan Novak, Karlheinz Peter, Kasper Rossing, Joost Schanstra, O. John Semmes, Dan Theodorescu, Visith Thongboonkerd, Eva M. Weissinger, Jennifer E. Van Eyk, Tadashi Yamamoto
Published Online: 22 Jan 2007
|Comment from:||Dr. Martin Latterich|
|Short affiliation:||Faculty of Pharmacy, University of Montreal, Montreal, Canada|
|e-mail:||firstname.lastname@example.org Date: 28 March 2007|
I read, with great interest, the Viewpoint article by Mischak et al., since my laboratory focuses on research in clinical proteomics. Specifically, we are conducting, together with clinical researchers and biostatisticians, several large scale clinical biomarker initiatives (cardiovascular disease and prognosis, chronic rhinosinusitis, and cancer). Our research hopes to ultimately enable a personalized medicine approach together with our collaborators in the area of pharmacogenomics. In addition, I am Editor-in-chief of Proteome Science. We often encounter some of the issues and decision making process mentioned in the article when we decide whether to publish or reject a clinical proteomic study.
The article resonated well with me in terms of outlining a need for a mechanism of involving a multidisciplinary team composed of clinicians, mass spectrometrists, statisticians, and biologists in such projects, as rigorous study design, sampling, assay development, data analysis, and interpretation are critical for the success of these initiatives.
However, there are several key areas where my opinion differs:
1. Setting adequate standards
Personally, I think the field has not matured sufficiently to impose standards at this time. There are numerous technical and conceptual hurdles that need to be conquered first by the community before understanding what best practices are. While it is important to have scientifically sound guidelines related to study design, sampling and interpretation, there are too many avenues such studies can take in reality. What is more important in the short term is to evaluate different technical platforms to define their advantages and limitations for distinct sample types. Very often, I find that comparative clinical studies are designed with accessibility of platforms, rather than optimal platform for a given sample in mind.
2. Cohort size
There is a cost-benefit relationship between study design, statistical
power, and availability of well-characterized disease material. A well designed clinical study would compare healthy and diseased individuals separated by age, gender, ethnicity, and lifestyle, with between 25 and 50 individuals per group. However, with very few exceptions, academic resources are not available for multi-million dollar studies of this scale.
In other instances, the occurrence of disease may be so rare, that a proteomic biomarker discovery effort would rely on the comparison of few individuals with that of close relatives.
Does this mean that smaller studies should not be publishable or even conducted? My personal opinion is that as long as studies are performed in accordance with good experimental practices, and if the studies list sufficient detail to enable others to reproduce the original findings, these studies should be publishable, as long as their findings are not over-interpreted. The value of such papers lies in the benefit to the scientific and patient communities, as it may help the development of biomarker validation and therapeutic strategies for when more patients become available in future.
3. Standardization of sample procurement
One of the key steps that introduces variance in a proteome analysis is the sampling method, and the rigor of the protocol by which samples are obtained. In my professional opinion, this area should receive most attention in terms of setting standards, because if sampling is performed according to slightly different sampling protocols, studies are no longer comparable due to sampling-induced variance. This is a major issue, especially when comparing plasma from individuals, because plasma samples degenerate quickly, when not properly collected and proteolytically stabilized.
4. Orthagonal methods of validation
It is easy to profile and compare patient proteomes to that of a control group, and to list disease "signatures". The real challenge in utility lies in the clinical validation of such profiles and biomarkers. For a clinical proteomic study to be meaningful, significant effort has to be spent on the development of a validated orthogonal assay (e.g. ELISA) to clinically validate the putative biomarker(s) discovered. This is both financially and time-wise a major undertaking, since it involves a new clinical study design, cohort recruitment and sampling, production of expensive reagents, and selection of an optimal assay platform. I think that any "biomarker" claim should be evidenced by a validated clinical assay, albeit this may delay publication of proteomic studies by several years.
5. Public database repository
Unless a rigorous, controlled multi-center study has been designed to allow identical sampling and analysis under identical conditions, I would not compare data sets from different groups, as the danger lies in over-interpreting any leads discovered by comparing data sets that may harbor sampling-induced variance. Instead, I would focus on the standardized collection and archiving of well-characterized patient material first, as this is as important to the success of such studies.
|Comment from:||Dr. Robert Moulder|
|Short affiliation:||Turku Centre for Biotechnolgy, University of Turku, Turku, Finland|
|e-mail:||email@example.com Date: 06 March 2007|
|Comment:||Thank you for forwarding the clinical proteomics article: This clearly provides a thorough, detailed assessment and recommendation of the necessary requirements for proteomic measurements that would meet the GCP level. This will surely provide a useful and frequently referred reference in field of study.|
|Comment from:||Dr. Xiaohong Li|
|Short affiliation:||Fred Hutchinson Cancer Research Center, Seattle, WA, USA|
|e-mail:||firstname.lastname@example.org Date: 05 March 2007|
|Comment:||The viewpoint article presents many important issues related to the clinical proteomics study. The recommended steps for clinical proteomics study presented in the article (table 1) are excellent steps for considering future clinical proteomics studies; and the suggested essential reporting requirements for publication (table 2) should be the common, if not mandatory, guidelines for reporting the clinical proteomic study results.
There are some points in the article, however, may need to be clarified: (1) the article states “As the proteome is far more extensive than the genome, it offers a richer source of potential biomarkers“. I am not sure this statement has been validated conclusively or not. Although the kinds of nucleotides used for DNA and RNA is fewer than the kinds of amino acids used for proteins, the number of possible ways for diseases development through DNA and RNA mal-format are probably in super-exponential magnitude (i.e. point mutation, chromosome loss, gain, translocation, copy number change, DNA methylation, DNA polymorphism, RNAi pathway, etc.). DNA/RNA may still be rich resources for biomarker development. In addition, DNA, RNA and proteins dynamically interact as a network system. Use both DNA and protein as biomarkers for clinical use are complementary, not mutually exclusive at this stage. (2) the article proposes that a clinical proteomic paper should not only report sensitivity and specificity, but also report receiver operating characteristic (ROC) curves for biomarkers performance evaluation. It is absolutely a constructive thing to report ROC, but the article does not give a guideline for ROC construction. Since ROC is a crucial index for evaluating biomarkers, but there are many ways to build ROC curves. In most cases (if not all), the shape of ROC curves depends on the authors’ specific algorithms (i.e. data pre-processing, peak identification, peak or feature selection, and prediction model construction) established for the biomarkers and the ways how the biomarker are computationally validated. There are various algorithms could be used for the analysis of a given set of proteomic data, or the same set of biomarkers; and many researchers use different approaches to statistically validate the biomarker prediction model (i.e. jack knife, leave some samples out cross-validation, bootstrap, etc.) and subsequently build ROC curves based on the results of model predictions. In addition, it is very common that in clinical proteomic studies, new algorithms such as data pre-processing, prediction models or modified prediction models will be developed. Therefore, it may be not enough just report the ROC curves in publications since they are algorithms and implementation methods dependent. I suggest that “Clinical Proteomics Community” may, for each major platform, set up (i) a few “Standard Datasets” for validating the algorithms in new publications (‘Standard Datasets’ here means they could be generally representative come out from a specific platform with representative biomaterials that have reasonable complexity); and (ii) choose two to three “Standard Algorithms” for dataset validation (‘Standard algorithm’ here means simple, robust, reasonably effective). Specifically, future clinical proteomic studies and publications may not only report the new algorithms and new results developed or discovered, but also briefly report (or report as online supplement part) the results from applying the new algorithms reported in the manuscript to the “Standard Datasets”; and also report the results from applying the “Standard Algorithms” to the new dataset reported in the manuscripts. It is not intended to match the data format of the future studies with the “Standard Datasets”, but just to test algorithms reported in manuscripts with “Standard Datasets”, and test the data in manuscripts with the “Standard Algorithm” in order to at least generate more specific information about the ROC curves reported and algorithms used in the future manuscripts. This might not only help to make the future results of manuscripts to be accepted by related researchers sooner, but also lead to a better cross-comparison among multiple studies.
|Comment from:||Dr. Angela M. Kaindl|
|Short affiliation:||Laboratoire de Neurologie du Development, UMR 676 Inserm-Paris, Paris, France|
|e-mail:||email@example.com Date: 26 February 2007|
|Comment:||Thank you very much for your email concerning the definition of "clinical proteomics" in the recent article by Mischak et al. One idea behind "clinical proteomic studies" is surely the identification of proteins that may be engaged in the pathomechanism of a disease or reparative processes without applying a hypothesis. However, since "lists of proteins" that differ between two conditions are not accepted for publication any more, and researchers are under pressure to publish (or perish), many researchers build a story around their data and pick out a couple proteins that will be studied in more detail. The negative effect of doing so is that we are taking a step back to a hypothesis-driven study with an over-interpretation of data. Of course, further studies need to follow that focuss on the effect of singular proteins - but these need to be seen separately of the primary "proteomic" study.|
|Comment from:||Dr. Gérard Branlard|
|Short affiliation:||INRA, UMR ASP-UBP, Clermont Ferrand, France|
|e-mail:||firstname.lastname@example.org Date: 23 February 2007|
|Comment:||We are working on Plant proteomics and I believe those standards will be profitable for all persons involved in that very important approach.
I will share that paper with my colleagues.
|Comment from:||Professor Jianyu Rao|
|Short affiliation:||UCLA, David Geffen School of Medicine and Jonsson Comprehensive Cancer Center, Los Angeles, CA, USA|
|e-mail:||JRao@mednet.ucla.edu Date: 22 February 2007|
|Comment:||Thanks for sharing me the excellent article - this is an important and timely article and will definitely help us in our study of biomarkers. A more clear discussion about how to select a panel of markers from proteomic profiling may be helpful.|
|Comment from:||Professor Kewal K. Jain|
|Short affiliation:||Jain PharmaBiotech, Basel, Switzerland|
|e-mail:||email@example.com Date: 22 February 2007|
|Comment:||Thank you for letting me have a look at the article. It is an admirable effort to set some guidelines for "Clinical research in proteomics" and not "Clinical Proteomics", which is a very broad term going beyond the definition proposed. Medicine has its own guidelines such as evidence-based medicine in addition to the regulatory oversight for new diagnostics and therapeutics. However, medicine is not all science and physicians use their own judgements in incorporating information emerging from biotechnology in understanding disease and modifying treatments. A considerable amount of data accumulated in proteomics databases, in spite of shortcomings, has been usefully applied in designing clinical research projects. It is desirable to set guidelines for studies and publications, but it is not always practical to implement these. It often depends on a discriminating reader to evaluate a publication.
The proposed guidelines will supplement the existing general guidelines for clinical trials, biomarker validation and molecular diagnostics research with special reference to use of proteomics.
|Comment from:||Dr. Emanuel Carrilho|
|Short affiliation:||Universidade de São Paulo, Instituto de Química de São Carlos, Departamento de Química e Física Molecular, São Carlos, SP - Brasil|
|e-mail:||firstname.lastname@example.org Date: 22 February 2007|
|Comment:||This article comes handy in perfect timing since my group is starting to be more active in the clinical proteomics. I have 3 students that will start right away following such guidelines.|
back to Viewpoint Articles