Impact factors...in defence of "Nature" and "Science"

More often than not, when the corrupting influence of impact factors on science is discussed, fingers are pointed at Nature and Science, as if these two scientific journals invented impact factors and as if they are the main culprits in debasing science. This is not even remotely true. Rather, the finger should be pointed at the academic community exclusively and on no one else. In fact, there are even limits as to how much the academic community can be blamed, as I argue in my next post.

Nature and Science, and especially the former, are the best known and most sought after of what has come to be known as the “glamour” journals in science. There are, of course, other “glamour journals”, as well as ones that aspire to that status, but none has reached quite the same status as these two. It is therefore perhaps not surprising that they should bear the brunt of the blame.

But it would be hard for even the enemies of these two journals not to acknowledge that both have done a remarkably good job in selecting outstanding scientific papers over so many years. Journals do not gain prestige status by impact factors alone; if they did, their prestige wouild fall, and their impact factors along with it. I myself have little doubt but that the editors of both journals are hard-working, conscientious people, striving to do the best by themselves, by their journal and by science. One way of measuring their success is through impact factors, which is a guide to how often papers in the journal are cited. Impact factors are blind to quality, readability, or importance of a paper. They are simply one measure – among others – of how well a journal is doing and how wide an audience it is reaching. One could equally use other measures, for example the advertising revenue or some kind of Altmetric rating. Impact factors just happen to be one of those measures. And let us face it, no editor of any journal would be satisfied with low impact factors for their journal; if s/he says otherwise, s/he lies. The un-ending search for better impact factors really boils down to human behaviour - the desire to convince oneself and others that one is doing a good job, to esteem oneself and gain the esteem of others. Editors of journals are no different. Like the rest of us, they aspire to succeed and be seen – by their employers, their colleagues and the world at large – to have succeeded. Is it any wonder that they celebrate their impact factors?

To the editors of these journals – and to the rest of us - impact factors are therefore a guide to how successful they have been. I see nothing wrong with that, and find it hard to blame them for competing against each other to attain the best impact factor status. In other words, there is nothing really wrong with impact factors, except the uses to which they are put, and they are put to undesirable uses by the academic community, not by the journals.

In spite of the sterling service both have done to science, by publishing so many good papers, it is also true that they have published some pretty bad ones. In fact, of about the ten worst papers I have read in my subject, in my judgment one (the worst) was published in Nature Neuroscience, one in Nature, and one in Science.  I have, as well, read many mediocre papers in these journals, as well as in others aspiring to the same status, such as the Proceedings of the National Academy of Sciences and Current Biology. This is not surprising; the choice of papers is a matter of judgment, and the judgment made by these journals is actually made by humans; they are bound to get it wrong sometimes, and apparently do so often. By Nature’s own admission in an editorial some time ago, there are also “gems” in it which do not get much notice. Hence, not only does one find some bad or mediocre papers in these journals but un-noticed good ones as well. Retraction rates in both journals are not much worse or better than other journals although retraction rates apparently correlate with impact factors, the higher the impact factor, the more frequent the retractions. But it would of course be entirely wrong to blame the journals themselves, or their referees, if they publish papers which subsequently have to be retracted. The blame for that must lie with the authors.

“Send it to a specialized journal” (euphemism for “Your paper won’t help our impact factor”)

I recently had an interesting experience of how they can also be wrong in their judgment, at least their judgment of the general interest in a scientific work (of course the more the general interest, the higher their impact factor is likely to be). We sent our paper on “The experience of mathematical beauty and its neural correlates” first to Nature, which rejected it without review, stating that “These editorial judgements are based on such considerations as the degree of advance provided, the breadth of potential interest to researchers and timeliness” (somewhere in that sentence, probably at “breadth of potential interest”, they are implicitly saying that our paper does not have the breadth of potential interest – in other words will not do much to improve their impact factors). We then sent it to Science, which again returned it without sending it out for review, saying that “we feel that the scope and focus of your paper make it more appropriate for a more specialized journal.” (Impact factors playing a role again here, at least implicitly, because, of course, specialized articles will appeal to a minority and will not enhance the impact factor of a journal, since they are also likely to be cited less often and then only by a minority).

Finally, going several steps down a very steep ladder, we sent it to Current Biology, which also returned it without sending it out to referees for in-depth review, writing that “…our feeling is that the work you describe would be better suited to a rather more specialized journal than Current Biology” (my translation- it will do nothing for our impact factor since only a limited number of workers are likely to read and cite it).

The paper was finally published in Frontiers in Human Neuroscience (after a very rigorous review). Given that this paper has, as of this writing, been viewed over 71,000 times in just over 2.5 months, and that it has been viewed even in war-torn countries (Syria, Libya, Ethiopia, Iraq, Kashmir, Crimea, Ukraine), it would seem that our article was of very substantial interest to a very wide range of people all over the world;  very few papers in neuroscience, and I daresay in science generally, achieve the status of being viewed so many times over such a brief period.  On this count, then, I cannot say that the judgment that the paper should be sent to a specialized general or that its breadth of interest was potentially limited inspires much confidence.

“We only want to publish the most influential papers”

It is of course a bit rich for these journals to pretend that they are not specialized. I doubt that any biologist reading the biological papers in Nature or Science would comprehend more than one paper in any issue, and that is being generous. In fact, very often what makes their papers comprehensible are the news and views sections in the same issue, a practice that some othert journals are taking up, though somewhat more timidly. By any standard, Nature and Science and all the other journals that pretend otherwise are in fact highly specialized journals.

Be that as it may, they are only pursuing a policy that many other journals also pursue. Consider this letter from e-Life, a recently instituted open access journal, which I have seen being written about as if it is a welcome answer to Nature.

Well, they returned a (different) paper I sent within 24 hours, after an internal review, saying that “The consensus of the editors is that your submission should not be considered for in-depth peer review”, adding prissily “This is not meant as a criticism of the quality of the data or the rigor of the science, but merely reflects our desire to publish only the most influential research”, apparently without realizing that a research can only be judged to have been influential retrospectively, sometimes years after it has been published. But what does “influential” research amount to – one which is cited many times, thereby boosting – you guessed it – the impact factor of the journal. Indeed, e-Life (which has also published some interesting articles) even has a section in its regular email alerts that is intended for the media – which of course help publicize a paper and boost – you guessed correctly again – the impact factor!

So why single out Nature and Science, when so many journals are also pursuing impact factors with such zeal? It is just that Nature and Science are better at it. And their higher impact factors really means that the papers they select for publication are being cited more often than those selected in other journals with aspirations to scientific glamour.

So, instead of pointing fingers at them, let us direct the enquiry at ourselves, while acknowledging that both journals, in spite of all their blemishes and warts, have done a fairly good job for science in general. 

In my next post, I will discuss why impact factors - however repellent the uses to which they are put by us - are here to stay.

Art and science meet up, sort of...

Some time ago, I wrote about an empty canvas by Bob Law, entitled Nothing to be Afraid Of, which was to be auctioned for an estimated £60, 000. Law was described by the head of the contemporary art department at the auction house as the "most underestimated and overlooked minimalist artist in Britain...who didn't get the recognition that he deserved". In his painting he had apparently "... applied the seductive idea of nothing to a canvas, and asks the viewer to reflect”.

A somewhat puzzled David Hockney was reported as saying "It seems to me that if you make pictures there should be something on the canvas".

In the end, the empty canvas was never sold, at least not at that auction.

Now, I have just read in Real Clear Science about the shortest paper ever published.

It is entitled "The Unsuccessful Self-Treatment of a Case of Writer's Block" by one Dennis Upper.
The paper is an empty page. The referee's comments are reproduced below the empty page and read as follows:

"I have studied this manuscript very carefully with lemon juice and X-rays and have not detected a single flaw in either design or writing style. I suggest it be published without revision. Clearly it is the most concise manuscript I have ever seen-yet it contains sufficient detail to allow other investigators to replicate Dr. Upper's failure. In comparison with the other manuscripts I get from you containing all that complicated detail, this one was a pleasure to examine. Surely we can find a place for this paper in the Journal-perhaps on the edge of a blank page."

There is nothing on the page -- and yet "it contains sufficient detail to allow other investigators to replicate..."

Bob Law asked the viewer to reflect by applying "the seductive idea of nothing to a canvas"

Both scientists and artists can now, in the absence of all detail, create their own details.

So science and art do meet, sort of, don't they? After all, who can deny the similarity here?

Maybe someone should ask the auction house to sell a copy of the paper (preferrably signed by Dennis Upper) alongside Bob Law's empty canvas.

That will be a true meeting of art and science - united under money.

The question is: which one will fetch the higher price?

Wonderful, transient, art in the snow

 

The British artist, Simon, Beck has created some memorable art over snow at a ski resort in France, some of which can be seen here. The pictures are attractive to look at but they must be more exciting to see for real.

With every new snowfall, the creations will gradually change and then disappear altogether. The art is therefore, of necessity, transient and has, therefore, transience as an added element. I wonder whether these creations, and their transience, will not be even more appealing to those of a Japanese culture, which emphasizes transience (wabi) as a feature of beauty.

Perhaps some contemporary art gallery should buy good quality photographs of these creations, to exhibit permanently what is only transient.

Just a thought!

The great Stephen Hawking

-->

At a recent event to launch the exhibition on the Large Hadron Collider at the Science Museum in London, the great Stephen Hawking made what must seem to many an unusual declaration. He said, “Physics would be far more interesting if [the Higgs boson] had not been found”. Physicists would then have to re-think many of their fundamental ideas about particles and the forces that bind or repel them.

By saying so, Hawking was displaying both the qualities and perhaps the failings of scientists. Scientists, or at least the great ones like him, love the process of solving great and difficult problems. The solution may be quite marvelous and exciting to think about; it may even be very moving. But, once solved, it ceases to be a problem, which the enquiring mind needs.

So, what Hawking was saying, it seems to me, is that if the Higgs boson had not been found, the problem would have persisted and exercised and concentrated minds, which is what scientists like so much.

This of course is very distant from those who wish that a problem should never be solved, because they fear the results. Some have written of their fear of work on the neurobiology of love, because it will “de-mystify” it; others have written, of neuroesthetics, that they would find it unwelcome to learn what happens in their brains when they view a work of art or listen to music. Hawking wants to learn; they don’t. If Hawking prefers that the Higgs boson had not been found, it is simply that he relishes the process of discovery. He is not fearful of the results; they are.

Why, then, should this also be a failing. I think it is because lesser scientists (and let us not under-estimate the degree to which scientific progress also depends upon lesser scientists) can easily be distracted from trying to solve great problems into solving relatively minor ones, precisely because they love the process of solving problems! I have seen it happen many times.

But there are of course many problems that remain in physics and astronomy. And Hawking is hoping that physicists will move on to solving even grander problems about the nature of our universe.

Hawking is not afraid of de-mystification.  Not at all. The mark of a real intellect.

The paradox of Shunga

The quite wonderful exhibition, Shunga: sex and pleasure in Japanese art  at the British Museum in London, carries with it a surprising paradox or contradiction, which no one has so far been able to explain to me adequately.

Japanese culture in general emphasizes the unstated and the understated, leaving much to the imagination. Yet Shunga art, which is basically erotic art, is the exact opposite. Here, almost nothing to do with the genitals is left to the imagination; instead they are given prominence, the size of the organs more often than not exaggerated beyond reasonable dimensions. 

Yet, in spite of this prominence, most of the rest of the body is covered up in many, if not most, depictions of sexual encounters; in many it is the genitalia alone that are exposed. There is of course, also something of the artificial in these works; couples make love with their clothes on; the hair is usually immaculately coiffed, in some a lady is having her hair combed while having intercourse while in others there are spectators, including children, witnessing the scene.

Why would a culture that has traditionally emphasized the understated produce work that is anything but under-stated? Some Japanese friends have told me that Shunga is nothing but pornography. I do not believe it. In spite of the fact that they may have been used as stimulants or as props for sexual pleasure, these are works of art as well. It is the brilliant depiction of interiors, the wonderful colour combinations, and the immaculate detail with which clothes are represented that turns them into visually pleasurable works. Indeed, it may be said that the genitalia are in fact often a distraction from the rest of the work, especially the depiction of the graceful women in the Shunga work of Kitigawa Utamaro. If  “art is fantasy”, as a quote at the exhibition proclaims, then it is those graceful figures that invite the viewer into a world of fantasy, not the prominently exposed genitalia. A critic once wrote that the sexual figure in Boticcelli’s Birth of Venus is not the naked lady but her richly dressed companion to the right, presumably meaning that it is the latter who draws the viewer into a world of fantasy. Maybe the great masters of Shunga art were trying to balance the explicitness of their images with depictions that allow a world of fantasy and imagination to come into play, all in one.

Shunga was apparently not legal in Japan for very long periods, though tolerated throughout and popular with all levels of society. It is, I gather, still frowned on in Japan. Indeed, I am told that, in modern-day Japan, adult movies in hotels often blur the genitalia – in striking contradiction to Shunga art of earlier times. And there is the contradiction: explicit pornographic films that blur the genitalia on the one hand (perhaps in keeping with the understated in Japanese culture), and great art that is implicit in everything but the genitalia (quite unlike the understated characteristic of Japanese culture).

The shocks of Francis Bacon

 

Francis Bacon claimed that he wanted to give “a visual shock”, and his paintings over the decades never seem to have departed from that aim. One of his first exhibitions, in New York, was described as a “chamber of horrors” and Margaret Thatcher, perhaps echoing the views of many outside the art world, once described him as “that man who paints those horrible pictures”. As I understand it, most people (even those who admire his painterly style) would prefer not to have his paintings hanging in their living rooms.

Last week, his three-panelled painting, entitled Three Studies of Lucian Freud, produced another shock – a financial one. It fetched a record price in New York, being sold for the sum of $142.4 millions. What is it that attracted buyers to spend so much (the bidding started at $80 million)?

I believe that Bacon subverted the brain’s normal representation of faces and bodies, which is what turned his pictures into shocking displays. The brain, it seems, cannot easily adapt to departures from what constitutes a normal face; it cannot adapt easily to the disfigured faces and bodies that Bacon specialized in, as a means of making images of the violent reality which, according to him, was so prevalent in the world. Hence the enduring shock effect that he produced.

Most of the discussions I heard and articles I read on this sale revolved around the topic of money. It is not that buyers were only speculating. Rather “deep-pocketed” buyers were also ready, it seems, to splash out considerable sums to buy paintings for their national museums or their homes. I am inclined to the view that when it comes to spending such vast sums, the long-term value is naturally important but cannot be the only or even dominant factor. So what, beyond the prestige of Bacon, drove prices so high? How could paintings reviled through the use of phrases like “horrors” or “mutilated corpses” or “extremely repellent”, which so many (including one on the radio last week) declared they would rather not see hanging in their living rooms, be so much sought after.

Perhaps we have a very deep-seated fascination with horror, especially when it is so evocatively depicted. Perhaps those who yearn to view such paintings are an infinitely more sophisticated and refined, indeed artistic, version of those who jam the roads on their way to see a crashed plane. There are, of course, huge artistic qualities to Bacon’s work – they are formally masterful works, with a quite spectacular, and often unusual, combination of colours. But the fact remains that they also depict mutilated and savaged faces and bodies – viewing of which almost certainly stimulates strongly sub-cortical centres such as the amygdala and the insula, which seemingly respond to fear and horror. And let us not forget that Bacon once said that he was not appealing to the intellect: “I make paintings that the intellect cannot make” he once said, which also implies that he was appealing to something more primitive in his work. In his quite wonderful book on Francis Bacon, Michael Peppiat says that Bacon’s aim was to deliver a visual shock before things got spelled out in the brain (or words to that effect). Perhaps, combining the aesthetically pleasing colours with the mutilation that he so consistently depicted makes the latter more palatable – and even pleasing. The more so if one knows that such a combination is a good place to park one’s money.

Academic violence

 

One is always somewhat surprised when academics who, in the words of HL Mencken, are generally as “harmless as so many convicts in the death house”, turn to violence. In general, academics dislike violence and prefer to pursue their trade peacefully, although there are many examples of verbal violence. I know of an English university department which speakers are reluctant to speak at because of the extreme verbal violence of one member there.

Yet it is surprising when this violence escalates to the level of arms. The BBC reports one such incident in which an argument about the German philosopher Immanuel Kant escalated to such levels that it ended by the use of rubber bullets fired by one protagonist against another. What the bone of contention was is not recorded. It could have been the “a priori synthetic” or the “categorical imperative” or perhaps the “transcendental synthetic”. At any rate, one of the protagonists was charged with causing grievous bodily harm.

Kant himself would probably have been very surprised. His book, Critique of Pure Reason, apparently sold only five copies when first published, of which two were purchased by himself (I cannot vouch for the accuracy of this story, which I read somewhere years ago). He was in general a very peaceful man whose habits were so punctual that housewives apparently set their watches by when he went to work and when he returned. The French critic Rémy de Gourmont marveled that a man like Kant who had neither wife nor mistress, who died a virgin (as Gourmont believed) could have written a book on the metaphysics of morals!

Yet, violence in academic circles has been recorded before (I mean real violence, not the verbal one, which is very common). There is, for example, the story of Pierre Marie, an eminent French neurologist, who accused another eminent French neurologist, Déjerine, of doing science as some play roulette. But, upon being challenged to a duel, Marie wisely chose to retract his accusation.

On one occasion, I was told not to mention 40Hz when giving a seminar if a certain gentleman was in the audience, for fear that he may suffer a heart attack. I wisely obeyed. But I am told that he later died of a heart attack anyway.

Perhaps it is only fear that keeps academics from resorting to real violence. I know of stories of one German physiologist saying of another, “Now that I have shown that he cannot use a slide ruler, I intend to take no further notice of his work”, while another accused a colleague of “auto-plagiarizing”. I can well imagine such incidents boiling over and resulting in - well, the firing of rubber bullets, at least.

It all goes to show that the dispassionate academics, searching for truth in their ivory towers, may not be impervious to these human instincts, just like the rest.