Clinics in Colon and Rectal Surgery 2017; 30(04): 259-263
DOI: 10.1055/s-0037-1604254
Review Article
Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

Changing the Rules of the Game: How Do We Measure Success in Social Media?

Aisling M. Hogan1, Desmond C. Winter1
  • 1Department of Surgery, St Vincent's University Hospital, Dublin, Ireland
Further Information

Address for correspondence

Desmond C. Winter, MD, FRCSI, FRCS(Gen)
Department of Surgery, St Vincent's University Hospital
Elm Park, Dublin 4, D04 T6F4
Ireland   
Email: deswinter@svcpc.ie   

Publication History

Publication Date:
12 September 2017 (online)

 

Abstract

Ours will be the generation proud to say we shifted the sands of educational deserts by open access and proliferation, seeding of data sharing, and watering grassroots research in resource-compromised environments. Universal “social” media is defining features of modern professional life that provide powerful modes of knowledge acquisition/sharing to that end. Altmetric and other measurements stratify academic communications according to this alternate, online media presence (not academic penetrance). Are they meaningless, self-absorbed integers, or reliable yardsticks of scientific and educational prowess? Far beyond this trite, patronizing question from the minds of outdated, terrified technophobes, the real impact of “social” media is not narcissistic solipsism. Instant dissemination of contemporary surgical controversies on a truly global level drives improved (or at least reflective) health care for all. While a numerical assignment of value according to views, “likes,” impressions, or “retweets” may seem meaningless to cynical, established academics, the impetus for universal improvement is self-evident. Electronic data and opinion sharing may not balance the inequity between low- and high-income countries, but it keeps it in perspective. The best way to shift desert sands is to blow on them constantly.


#

Just as children compete to build better sandcastles on the beach, the social comparison theory, popularized by Festinger in 1954,[1] describes the need of individuals to gain accurate self-evaluation by comparing themselves to others. Surgeons, physicians, and academics are prime examples of groups that highlight this intrinsic human need. Perhaps, it is fueled by an inherent competitive drive but comparison to peers fosters and nurtures the much-described stereotypical surgical ego.[2] In few other groups, is mutual comparison sought with such vigor, from impact factors and H indices to “results” and “outcomes.”

Technological developments have always influenced scientific interactions. Many communication revolutions have been overcome from the birth of the postal service in the mid-19th century to the induction of the Computer Science Network (an embryonic form of the modern day internet) in 1981 and the most recent meteoric rise of “social” media in the past decade. Its widespread availability renders it an extremely powerful tool for communication and interaction among the global scientific community. Generic availability of novel scientific data on “free” sites could and will revolutionize the practice of surgery on a global scale.

The power of social media is currently more than ever predicted and the implications of instant dissemination of surgical literature in the global sphere are vast. It has been demonstrated that highly “tweeted” articles are more likely to become highly cited.[3] Thus, social media drives interested parties to a more conventional knowledge source (in search of full text articles on journal Web sites). A majority of journals are utilizing this tool to increase international readership and penetration. It appears then that gauge of twitter activity surrounding any scientific publication could be used by publishers as a predictor of future effect on journal impact factor. Perhaps, dissemination of bite size messages on social media sites will replace the conventional peer-reviewed process and publications will be informally and pre-emptively selected based on popular vote? This remains to be seen, but it is clear a new system of evaluation is needed to judge success in the digital age, where social media has completely changed the rules of the game.

History

Surgeons and scientists are experts in balancing personal and professional applications of social media.[4] [5] The process of peer review has evolved, and although it still has relevance in its traditional form, the second process (that of online review by the broader community) which frequently occurs within hours of publication carries far more weight. The concept of open communication in science is not new.[6] The acclaimed sociological historian Merton described the “norms” of scientific communication as Communism, Universalism, Disinterestedness, and Organized Scepticism or “CUDOS” in 1979.[7] It could be argued that Facebook, Twitter, Blogging, Instagram, and Symplur epitomize the pillars of Mertonian communication. The main barrier to information acquisition in the developing surgical community is monetary, but in the case of social media, content is freely available to screen devices with internet or cellular access. Seventy-eight percent of Twitter users take place on mobile devices. Even in the poorest communities, a majority of professionals have access to smartphones and Wi-Fi, thus revolutionizing the process of continuous professional development through medical education. This reduces the need for international travel to conferences for engagement with contemporary surgical practice. Geographic, temporal, and monetary boundaries are removed. Expert discourse and interchange serve as an “online journal club” and critique of published literature is available almost instantaneously.[8]

The sands of scientific writing are shifting so fast that the landslide is almost perceptible. The relevance of impact factor (popular since the 17th century) is being undermined by the outdated, manipulable, and time-lagged nature of the metric. The metric is calculated yearly, based on a yearly average number of citations compared with recent publications. Higher impact factors have a greater relative importance than smaller ones. Examples of impact factors are The New England Journal of Medicine: 59.5, Diseases of the Colon and Rectum: 3.7, and Clinics in Colon and Rectal Surgery: 0.714.

Arguments against the use of the impact factor as a metric include that there are no viable alternatives and are simply the hollow sophistry of a threatened elite. Flawed as these surrogate markers of academia may be, they did provide a common language for mutual rating, albeit one that was self-serving, quasi-mathematical, and clumsy. Traditionally, simplistic comparisons could be made and conclusions drawn. Journals were weighted, authors bore a public value, and we all spoke the same language. Journal publications form only one peak in the mountain range of scientific communication. Every article is reviewed and critiqued instantaneously and opinions and criticisms are widely available (unlike the traditional model of anonymous peer review, where a small subset of reviewers determine the fate of an article—where the authors are not anonyms and thus bias or personal opinion may play a role in this process). In this new form of peer review (after online publication), the authors receive invaluable feedback, both positive and negative from a much wider audience. Will open access and a more transparent review process improve the quality of surgical publishing? Not necessarily, but there is wisdom in a crowd-shared approach. Knowledge was a fiercely protected resource in the 20th century. The 21st century will see a balancing of knowledge sharing on a global scale. It should be for the better.


#

Immediacy

Only a minute percentage of scientific communication occurs via an article medium and journals face being somewhat outdated before they emerge from the printing press. There are often significant delays from the time of data gathering to meeting/podium presentation to article submission, review, revision, and finally publication. These delays are being targeted by a new generation of medical professionals who grew up in a digital age where instantaneous access to free information is the new norm. The older cohorts of surgeons are being usurped by a new generation of communicators. Surgical literacy is evolving at a rapid pace. International communication is instantaneous and ubiquitous. Seventy-seven percent of all twitter accounts are held outside the United States. It is not unusual for a photograph of “the perfect specimen” to be tweeted and commented on from multiple continents before ever it is seen by a pathologist. This echoes international political reporting where events are presented with virtually no lag time, thus encouraging the need for immediate receipt of all news stories—either political or surgical. Measuring academic success in this era represents a new challenge. If an academic idea or statement is seen and “retweeted” by a million individuals worldwide, what weight should this have in comparison to a peer reviewed first or senior author publication (that may only be read by the authors and the editor, according to a study done on article in the 1980s).[9]


#

Social Media and Continuous Professional Development

There is an emerging need for evolution in the awarding of points for continuous professional development. Time spent debating current contentious issues in the public domain should and will be rewarded by the relevant bodies. It is at least equivalent (and usually educationally superior) to participation in conference discourse or publication of scientific communication. Never have educational resources for medical students and doctors in training been so contemporary.[10] Comparison of international practice has long been recognized as vital in the process of audit and this has been simplified by the popularization of social media. One hundred million users login to Twitter daily. There are 320 million monthly active users and 500 million tweets are sent per day. Thus, the impact of this mode of communication on global surgical practice from high- to low-income areas and back is immeasurable.

The responsibility of the boundaries of ethics on social media is yet to be defined within the surgical community.[11] The question of specific consent of sharing of photography has yet to be explored. Although an individual specimen is unlikely to allow identification of an individual by any physical attributes, there are definite concerns about individuals forming temporal relations and thus connecting a social media photograph to a specific patient.

It is impossible to ascertain whether the impetus for such social media activity is the drive to educate and foster meaningful discussion or (like a majority of Facebook activity) a cry for endorsement and appreciation. Perhaps, opening and virtual chairing of international discourse on controversial topics should be awarded with a point system of sorts. There is no doubt but that following specialty leaders proves educational and economic of time and a majority of practice changing publications can be succinctly presented in 140 characters. While traditionalists voice concerns that many nuances are lost in brief communications of scientific findings, in truth, a majority of readers draw conclusions based on abstracts alone. Furthermore, the subsequent traditional communications or letters to the editor do not receive nearly the same attention as the original article. The popularization of postcard style visual abstracts or infographics allow assimilation of multiple articles within several seconds. While this undoubtedly has a positive effect on surgical education, there is no single figure, ratio or coefficient that can easily quantify it. All that we can conclude is that simple visual presentation of important surgical findings will disseminate widely, stimulate educational discussion, drive readers to the source content, and up everyone's game.


#

Impact in the First World

Impact factor is not a surrogate marker of penetration nor does it attempt to be. The present generation of university students has grown up in an era of instant gratification. An immediate “like” or a “retweet” is held in far higher regard than a rise in impact factor (up to 2 years later). Universities must embrace this shift in perception and begin to recognize and reward achievement appropriately. Otherwise, reliance on traditional appraisal methods will lead to inaccurate quantification of academic throughput. Real-time assessment of global impact of scientific output is, however, not a simple concept. There are various intricacies that need formalization before this process could be instituted—definition of what constitutes a scientific communication, potential weighting of authors based on past publication history, and valuation of a retweet, a like or a Web site hit.

Altmetrics[12] is a relatively new science but has potential to offer guidance as to where to begin in the weighting or validation of social media use. It tracks articles (referenced by a hyperlink) on various social media sites and a manually maintained list of Blogs. In time, the science of Altmetrics may be expanded to enable detailed and comprehensive analysis of individual researchers, bloggers, vloggers, Facebook, and Twitter users and to quantify their contribution to the wider scientific community. Conceived in 2010, it has attempted to address the quantification of success in social media. It analyses data of articles from supported publishers only and attempts to apply a weighting system where media felt to be further reaching are weighted heavier. It makes no effort to comment on or quantify the quality of the data itself, but its aim is to assess and place a numerical value on the “reach” of the article in the media. Articles cited in conventional news stories are given a weighting of eight times that of those cited on twitter. In this regard, the Altmetrics is outdated just as it becomes relevant. It is doubtless that the most prolific journal readers and “citers” spend far more time doing just that than on conventional mainstream news, so it is time for evolution.[13]

Definition of success is more complex than ever. Should there be a formula to incorporate the well-validated concepts of impact factor and H index with the emerging Altmetric value and number of social media followers? The concept of professional ranking leads to many unanswered questions.[14] Does a following from a field leader equate with a follow from a friend? Should we consider a retweet to be a citation? And if so, every retweet? Although surgical journals are widely available, they are rarely accessed by patients or the lay community. This is not the case with social media. It has been suggested that access be tiered with certain content available to medical professionals only. However, this is at odds with one of the great strengths of social media—its universal availability, and thus, with Mertonian rules of scientific communication.

Valid questions have been asked regarding how appropriate the use of generic social media sites is and whether there should be a separate platform for instant communication within the medical community. In that instance, stratification of success would be easier (quantifying Symplur views). It is likely that definition of such a value would require intensive collaborative assessment by mathematicians and statisticians but, undoubtedly, we will be presented with a formula to assess overall contribution to the worldwide scientific community within this decade. This will enable robust comparison of institutions, individuals, journals, and articles.

Publons and others have eased the scientific community into methods of scoring their online academic performance (i.e., journal referee tasks). Similarly, we have tried to score authors and journals for a long time without a good system. Several Web sites attempt to link the surgical community through publications and thus rank individual authors, often assigning a seemingly arbitrary number to each.[15] However, the multiplicity of values assigned to every academic (on various sites) renders all of them meaningless. The merit in having a single value assigned to an author or journal is undisputed and allows indulgence in the generic need for social comparison.


#

Impact in the Developing World

Dissemination of surgical knowledge in the first world is easy. Access to institutional journal subscriptions, physical presence of highly trained experts, and easy access to well-renowned teaching institutions enable education. However, the global surgical community is far more diverse than this, and many that practice outside of large institutions may not have the financial resources to maintain such widespread access and license to read full text journal articles. Any medium that is universally available to support those making decisions at the cold face with no access to other educational resources cannot be underestimated. The immediacy of social media means that it is perfectly placed to provide instantaneous continuous professional development to those with few or no other resources. Publishers have a stranglehold on contemporary surgical research and innovation, and perhaps, the very best measure of success would be quantification of availability in otherwise educationally deprived regions.

Ideally, all surgical communications would be open access for several reasons—removal of publishing biases, equity of access to education, homogenization, and optimization of practice. However, this is unlikely to occur in the multimillion dollar publishing empire. It is clear, however, that the rules of the game are changing. As first world surgeons, we have an obligation to ensure the diffusion of contemporary knowledge to our colleagues practicing in the developing world and contribute to universal equity in health care. Since social media drives those with easy access to journal Web sites and article downloads, one option is to use the revenue generated by publishers to make this content available in third world countries. This potential decrease in knowledge gradient would be the ultimate measure of success of social media and would represent a landmark in history as the antithesis of the 19th century resource stripping of the “third world” by the “first world” (as they were known in the self-indulgent, self-scored days of empires—narcissistic solipsism was alive long before Facebook). It is an opportunity to make reparation for previous injustices.


#

Weighting

The current system of peer review has been in place since the 17th century and it is universally acknowledged that change is necessary. Traditional markers of success in scientific publishing (citation index) refer to one author's reference of another's work. This includes an author's references of their own previously published work. Conventional online publication allows for relative universalization of the review process. It may be that dissemination beyond traditional academic circles enables less biased and thus more meaningful review. However, instead of adopting a simplistic view of discarding the old system and adopting a new one, it is perhaps advisable to direct efforts toward finding practical ways of incorporating the best properties of our old system into the new one. Several questions remain unanswered: How can we accurately track informal usage such as reading, sharing, and annotation (vs. just retweeting a link)? Is there a method of differentiating between articles' popularity on social networking sites due to their “tabloid” value versus those which are discussed due to genuine scientific merit?

Paradoxically, the potential for instant dissemination of data internationally could lead to greater reluctance of researchers to communicate ideas or results. Scientific meetings are awash with bloggers and unpublished data in the form of infographics frequently appear on social media sites before a speaker has reached his concluding slide. Subsequently, conference organizers often require attendees to obtain permission from speakers before blogging or tweeting. Preserving the integrity of science, and by extension society, in the era of Twitter and Facebook represents a significant challenge. In embracing technological progress, it is crucial to appraise new developments in communication with the same critical eye that is applied to laboratory data. Opportunity for expert discourse in real time enables open and unchartered critique of data. The opinions of experienced researchers and clinical practitioners are invaluable in guiding a trainee in interpretation of published work and drawing conclusions regarding it.

The introduction of a widely adopted weighting system of social media use would enable universal quality control. The Oxford grading system for levels of evidence (on the basis of study quality, imprecision, indirectness) endeavors to allow the scientific reader to immediately categorize findings prior to forming an idea on its quality. The mere publication of data in a peer-reviewed journal is perceived to ensure a minimal robustness and allows stratification of evidence. The concept of impact factor takes this one level further. However, in data or opinion presented on social media, there is no “filter.” Robust scientific findings of eminent leaders are as readily available to the general public as opinions of the entirely unqualified. The initiated recipient of information can easily distinguish these “levels of evidence,” but, with availability to the wider community, differentiation may be less clear. Thus, a simple figure (akin to the H index of conventional publication) would immediately allow the social media user to quantify the importance the data should be afforded.


#

The Present Is Already the Past

Universal social media is a defining feature of this generation of surgeons and academics. To its credit, it has embraced technology and used it to advantage but will this affect practice.[16] [17] Audit is a cornerstone of surgical development, but it is difficult to apply it to social media use because there is a merging of social and professional life with unclear margins. Quality assessment of information disseminated on social media is challenging to validate.[18] [19] To classify social media users into various categories, a complex, composite scoring system that belies the beauty of the internet would be required. Breaking down barriers to information sharing should be humanity's collective goal, not building sandcastle monuments to our achievement. For now, success continues to have many definitions. These are likely to continue to evolve as the world grows more accustomed to new ways of knowledge sharing.


#
#

No conflict of interest has been declared by the author(s).


Address for correspondence

Desmond C. Winter, MD, FRCSI, FRCS(Gen)
Department of Surgery, St Vincent's University Hospital
Elm Park, Dublin 4, D04 T6F4
Ireland   
Email: deswinter@svcpc.ie