Can AI Transform The Peer Review Process?

For scientists, the process of doing research is often much shorter than the process of getting any of their research published.

According to an article this week from Nature, an international scientific journal, peer reviewers make up only 20% of the world’s research scientists. With this, it is easy to see why this process can reportedly include significant delays.

What if, however, AIs were added into the mix?

Judging by the same article, two specific cases can help to illustrate how this has already begun to occur.

First, Nature mentions a software platform called StatReviewer, which specializes in indicating whether the research methods and statistics used by researchers are acceptable. Most importantly for our purposes, all of this is underpinned by an AI system. Beyond this general point however, the article actually fails to explain exactly how StatReviewer’s AI system plays into this process. Even so, the next case appears to do a much better job of clarifying how these situations typically play out.

UNSILO, which is a Denmark based AI firm, has partnered with one of the world’s largest peer review platforms. What Nature claims UNSILO is able to do for this platform’s reviewers is actually quite telling.

According to them, UNSILO summarizes each research article for the reviewers so they can completely focus their attention elsewhere. If you’re wondering what allows their AI to do this, it actually all comes down to a mixture of nature language processing and machine learning algorithms. For the purposes of clarity, just think of this as an AI system that uses a special sort of algorithmic framework to think outside of the box. In this particular usage of NLP, the AI would run one or more entity analyses to determine all of the words in the text that are semantically significant. From here, it appears that the system then uses these words to construct believable and easily understandable sentences.

If, like me, you are curious about just how believable and understandable these sentences are, check out a project like v*AI*lentine, which uses NLP to create Shakespearean sonnets.

While there seems to be a gap between what this type of technological framework can do and what humans can do, the similarities are still quite interesting to behold. Judging by the specific example of v*AI*lentine, even now, AIs can be poets though as Adweek says, their efforts end up looking more like those of kids who are just learning what poetry is. If you want to see exactly what I mean by this, take a look at the Adweek link below.

With UNSILO however, its’ technical abilities go a bit deeper than just using entity analyses to construct a text.

On top of running entity analyses, it reportedly acts in a similar way to the Turnitin service that most universities are now using. In the event that you’re not familiar with Turnitin, it analyzes a paper in terms of its similarities with every other source on the web. In the end, it provides a similarity score in the form of a percentage, in connection with a recommendation on whether or not the paper has been plagiarized.

In the end, however, even UNSILO’s sales manager stops short of claiming that their tool is meant to take over the editorial process. Reportedly, he was precisely quoted as saying that the system functions as an evidence gathering service for the editor, leaving the final judgment on the research up to the human worker in the end.

Thus, even though UNSILO can do some of the work for a peer reviewer, it still falls short of the traditional definition of being intelligent. In other words, as we mentioned above with entity analyses, this refers to the fact that AIs still operate within the boundaries of supervised learning. The team set very specific parameters for UNSILO’s AI before it went live, making sure to narrowly define its’ goals and what it should avoid doing.

Logically, once AI systems are able to break free of this mold, we will have truly reached the age of Artificial General Intelligence.

 

References:

https://towardsdatascience.com/an-easy-introduction-to-natural-language-processing-b1e2801291c1?gi=5870e8452f07

https://www.nature.com/articles/d41586-018-07245-9

https://www.ibm.com/support/knowledgecenter/en/SSWSR9_11.5.0/com.ibm.mdshs.wbuser.doc/topics/c_wbuser_entity_analysis.html

https://www.adweek.com/digital/this-ai-tool-generates-personalize-shakespearean-sonnets-to-give-to-your-love/

About Ian LeViness 113 Articles
Professional Writer/Teacher, dedicated to making emergent industries acceptable to the general populace