A Dynamic Simulation Model of Academic Publications and Citations
Bilge Kiiciik Nisa Giiler Burak Eskici
Bogazici University Bogazici University Bogazici University
Dept.of Industrial Engineering Dept.of Industrial Engineering Dept.of Industrial Engineering
34342 Bebek, Istanbul, Turkey 34342 Bebek, Istanbul, Turkey 34342 Bebek, Istanbul, Turkey
bilge.kucuk@gmail.com nisaguler@gmail.com burak.eskici@gmail.com
Abstract
In academia, the two main measures of research performance are publications and
citations. These two measures in a sense quantify the research success of scientists and academic
units. Perception of these performance measures can create pressures on researchers and cause
different behaviors in different conditions. The aim of this study is to examine the behaviors of
researchers in response to the dynamics of publication and citation pressures. A model including
faculty members in a department, their publications and citations has been constructed by using
system dynamics methodology. An important factor that determines citations for a paper is the
quality of the paper. Reputation of an academic unit is established as a result of citations that the
unit receives over time. There is an important feedback loop so that the reputation in turn
influences the citations the units will enjoy. A researcher, who has citation pressure on him,
would be forced to produce higher quality papers for getting more citations. On the other hand,
publication pressure would cause the researcher to produce lower-quality papers in higher
numbers, in shorter times. The main decisions of researchers are thus modeled through
allocation of researchers’ time in research activities and time devoted on each research. The
results obtained agree with our dynamic hypothesis and qualitative information about the
behavior of actual academic units.
1. Introduction
Academic knowledge proceeds by accumulation and an academician tries to make
contribution to this accumulation by publication. He aims to be the part of the common
knowledge and shares his work by publishing, but it does not mean that every publication means
a good contribution. To get citation in others’ papers has that indicator role. Citation can be
defined as the glue that binds a research paper to the body of knowledge in a particular field (5).
The journey is that somebody makes research and publishes it, and then others use it to make
new researches. It can be conceptualized as generating, distributing and consuming the scientific
knowledge (12). In this journey the key factor is to measure the contribution of a paper to the
field, because it is very important to decide what to be read, especially in the expansion of
scientific literature (7). The question is how the quality of a paper and an academician can be
measured. For this question the “scientometrics”, the science of measuring and analyzing
science, has emerged. Today, scientometrics have different kinds of measures all of which build
on number of publication and citation (4).
In addition to deciding what to read, scientometrics has emerged in order to quantify the
success of the academicians. The rapid expansion of the academic work has required an
objective measure and firstly the number of the publications considered adequate to decide
whether a researcher is successful or not. However, since the number of papers does not say
anything about the quality of the work, indexing — citation issues have entered the measurement
of science (4). Then there has emerged some publication citation balance measures (Like h-
index, g-index). Today, h-index, and average number of citation per paper are widely used
measures for an academician.
2. Problem Identification and Data Analysis
While in the past the reputation of the academician was important for personal
satisfaction; today it is a necessity to survive. Because academic career, position, tenure,
promotion, grants are all dependent on the reputation. The saying “publish or perish” explains
the situation. How to measure reputation has affected the evolution of the science heavily and
since for 50 years it is measured via number of publication and citation, today’s science
publishes with the question in the speed of progressing. Statistics compiled by the Institute for
Scientific Information (ISI) indicate that 55% of the papers published between 1981 and 1985 in
journals indexed by the institute received no citations at all in the 5 years after they published
and the conventional wisdom in the field is that 10% of the journals get 90% of the citations. If
the bottom 80% of the literature just vanishes, this confirms the suspicion that academic culture
encourages spurious publication. It is totally against the aim of the publication, but no longer it
represents a way of communication with scientific peers; rather, a way to enhance the status and
accumulate points for promotion and grants (5).
The type of the citation received is another issue in evaluation of citations. If received
citation is of international type, then it is logical to think that it is valuable, i.e. it contributes to
reputation more. As opposed to that, if a large portion of the citations are received from
colleagues in the same department, there is an ambiguous point about how to evaluate that.
Furthermore, if academic performance is evaluated by number of publications and
citations, is there a policy to be followed in order to maximize the performance? It is possible to
be very successful in one of the two. For example it is possible to increase the number of
publications without much care about quality. On the other hand, it is another option to have a
few publications that of high quality type. Is it better to have a balance between them?
Considering all these ambiguities and problems, it is true to say that there is not a specific
cause of this situation. It is a systemic problem. In this complex system the leverage points
should be found in order to design science policies for a better scientific progress.
To criticize our performance in an activity, most of the time, we compare ourselves with
the others; others, who are appropriate to be considered as a reference. In the academic world,
the “others” is the world average. An academician, whose “number of citations per paper” is
much lower than the average number of citations per paper in the world, would consider him as
poor in terms of citations. That will cause a pressure on him to produce papers which are more
likely to get citations. Similarly, comparing “number of publications” with the average number
of publications per faculty in the world, one would determine his performance to be satisfactory
or not. In order to be able to make those comparisons in the model, a data analysis is done.
Three main data are used in the data analysis part.
. Number of Publications
. Sum of times cited
. Number of faculty
For “number of publications” and “sum of times cited”, the data is collected from ISI Web
of Science which is a comprehensive database about publications and citations. It is an online
academic database provided by Thomson Scientific. It covers about 8,700 leading journals of
science, technology, social sciences, arts, and humanities. Seven particular fields are selected
from different countries that are selected to be representative of the world. These are
engineering, industrial engineering, biology, mathematics, physics, psychology and economics.
Moreover, countries, which are chosen, are USA, China, India, Russia, European Union
(Belgium, France, Germany, Italy, Luxembourg, Netherlands, Denmark, United Kingdom,
Greece, Spain, Ireland, Portugal, Austria, Finland and Sweden), Australia and Turkey. “Number
of publications” is of years 2000, 2003, and 2006. “sum of times cited” is for the publications
which are published in year 2000 and have been cited up to now.
For finding “number of faculty” values, a sample of about 50 universities is chosen for
each field. The universities are selected so as to represent the whole world. %60 of the
universities selected for each field are from USA, %20 from European Countries and %20 from
other countries. Data is collected from sample universities’ web pages for 2006 (6).
It should be emphasized that, almost all countries selected are developed countries and
their science figures are quite good. Besides, the universities selected are all in the top 500 list.
So the average values used in the model are obviously higher than the actual world averages.
The data analysis showed that different fields have different characteristics in terms of
academic publications and citations. Number of publications of a department in a field cannot be
compared to a department in another field unless the numbers are normalized. To see the
differences Figure 1 can be examined.
Avg. # of Publication per Faculty
2,50
2,00
1,50
1,00
_ at
0,00 |_|
2 | 8 8 3 g 8 B 2
= 5 4 8 a 8 3
g 32 @ S 2 5 3 s
i 3° o oO ao e 5 oO
S £ 2 8 S =
5 3 w
wu Ss a
51 44 53 52 52 53 51 50
Figure 1: Average number of publications per faculty per year
The world average number of publications per author in engineering is more than 1.5
publications per year where the same measure for economics is below 0.5. So, if a particular
measure of a discipline is to be evaluated, it must be compared to the worldwide average of that
measure in the discipline.
The formula used in finding average number of publications per faculty per year is as
follows;
Number of publications
Avg #of publications per faculty peryear= Number of Faculty
In our model, engineering department is selected to be studied. “average publication per
faculty per year in engineering” is 1.6 which is used in the model as a reference in determining
publication pressure. For finding “average citation per publication in the world in engineering”,
the following formula is used:
Sum of times cited
Average number of citations per paper= Nuntber of publications
In engineering, average number of citations per paper is 9.6. This is used in the model as a
reference in deter mining citation pressure. Figure 2 shows the comparison of Bogazici
University and world average for seven fields in terms of average number of publications per
faculty.
Bogazici University vs World Average
B Bogazici
Univ
@ World
Average # of Publication per
Figure 2: Average number of publications per faculty - Bogazici vs. World
In the model, the data obtained in this part is used as the initial values of the variables.
Additionally, the qualitative information as allocation of researchers’ time in research activities,
time devoted on each research, publishing time per paper are obtained by observing the
behaviors of researchers in Bogazici University Engineering Department.
3. Dynamic Simulation Model
The dynamic simulation model includes faculty members in a department, their
publications and citations. System dynamics methodology is used in constructing the model. The
aim is to examine the behaviors of researchers in response to the dynamics of publication and
citation pressure. Reserved time per paper, total research time, reputation and quality of papers
are included in the model as the main factors affecting the behavior. Complete stock-flow
diagram can be seen in Figure 3.
Figure 3: Stock- Flow Diagram of the complete model
Engineering faculty of Bogazici University is chosen to be examined. The initial
conditions, the number faculty and grand average values are obtained in the data analysis part.
Time unit in the model is quarters. 200 quarters (50 years) is examined in the simulation. Time
step (dt) analysis is done and dt is chosen as 1/8.
The model has an aging structure of the papers and the relation of citation part with this
structure. The stocks in the model without any detail can be seen in Figure 4. The four stocks
5
seen in the upper line represent the papers in different stages. The first stock in this aging
structure is ResinW (Research in Writing) stock and shows the papers which are in the research
stage yet. The papers are published and are started to be cited after approximately 3 years and
this transition stage is represented by the NewBornP stock which shows the published but not
citable papers. The stock PubPaper represents the papers which are published and being cited.
After staying in PubPaper for a long time depending on their quality, the papers become old
(13). The stock Obsolete P represents the papers which are published a long time ago and do not
get citation any more.
Publish
Resin) NewBomP. PubPap; ObsofeteTibsolete P
Research PublishR MatureR ObsoleteR
ExtC Intc
ActRbp
Rep AT
PerRep
ExtC\P EXICR Int R IntC\P
PerRepChange "| I
Figure 4: Simplified Stock-Flow model
IntC accumulates internal citations (citations from the same department of the author) and
ExtC accumulates external citations (international). Finally, PerRep is the perceived reputation
of the department all over the world. In the modeling of reputation, there is a first order
information delay.
The most important effect variables which are not shown in Figure 4 are shown with a
causal-loop diagram in Figure 5. In Figure 5 “total research time” is the researchers’ time in
research activities and “reserved time” is the time devoted on each paper. As seen in the causal-
loop diagram, reserved time per paper is affected by publication and citation pressures. When
citation pressure increases, a researcher would try to increase his citations by producing higher
quality papers. So, he would increase reserved time, i.e. he would spend more time on each
research. This would reduce average publication per faculty per year which would cause the
publication pressure..When publication pressure increases, a researcher would be forced to
increase his publications and so he would produce more publications in shorter times. There are
two negative feedback loops regarding reserved time and the pressures. So the faculty members
will come over these pressures by deciding on the amount of time per paper. Besides, total
research time is affected by publication pressure in that if there is pressure, amount of time
allocated to research activities would increase. This negative feedback loop will try also to
6
overcome publication pressure. Apart from these, there is an important and well-known positive
feedback loop between reputation and citations. If average number of citations per paper
increases the faculty will increase its reputation and if its reputation is high it will get more
citations.
+ total
research
Ve time =
average
publication Qe publication
Pressure per faculty
6 2F
pS,
citation EC quality
pressure _
<< average J
citation
per paper
reputation
Figure 5: Causal-Loop diagram
3.1. Formulations
3.1.1. Citation Pressure and Publication Pressure
Resir
PublishR
‘Avg CitiP
Gay Cit\P
Eff CitPressure
on ResT1P
‘Avg PublFly
Eff ResTime\P
on Quality
Gav g Pub\F\y
Eff PubPressuré
on ResT\P
Figure 6: Citation pressure and publication pressure in the model
One of the main effects in the model is the balance between citation pressure and
publication pressure.
Average Citation per Paper
ein) Ca Grand Average Citation per Paper
Average Publication per Faculty per Year
Publication P: =
rua ar Grand Average Publication per Faculty per Year
where grand average publication per faculty per year and grand average citation per
paper are variables that are obtained from world data in the data analysis part.
It is assumed in the model that, the time the researchers spend on each research depends
on the pressures on them. If the average number of citations per paper is lower than the world
average, than the researchers will feel a pressure to produce higher quality papers to get more
citations. For higher-quality papers, the researchers will need to spend more time on each paper.
The formulation of reserved time in the model is as follows;
Reserved time per paper
= Required time per paper*Effect of Citation Pressure*Effect of Publication Pressure
On the other hand if the number of publications of the researchers is much lower than the
world average number of publications in the particular field, the researchers will feel a pressure
to produce more publications. In order to increase the number of publications, they will decrease
time devoted on each paper. This will provide more publications which are lower-quality. The
effect formulations can be seen in Figure 7.
1500 1.000
Eh fons
weil PubPressur
CitPressure eed
on ResT\P ads
FOO 0100 : Ce
L__J
Va) (no ron
Ava Pub\F\y/Gavg_Pub\Fly
Figure 7: Effect of citation pressure and publication pressure on reserved time
Avg_Cit\P/Gavg_Cit\P
3.1.2. Total Research Time
faculty
Resinw
EffPubPressure TolpinesTiney
on TotResT Total Res Time
Figure 8: Total research time in the model
It is assumed that, if there is publication pressure, then in addition to decreasing the
reserved time, the researchers will also try to increase the total time they spend on research
activities.
Total Research Time
= pi ee
Heseareiy— Sacuny Reserved Time per Paper
Total Research time = Effect of PubPressure on Res Time*Total Res Time Normal
3.1.3. Quality
Eff Quality
on Citation
Eff Skill
on Quality
Eff Quality
on ObsT
Eff ResTime\P
on Quality
Figure 9: Quality in the model
Quality is one of the key effects in the model. Skill level of members of the department is
very important in the quality of the papers. It is assumed that, good-quality universities hire
researchers who produce good-quality papers. So, one of the indicators of the quality is the
overall skill level of the members of the department. In the model, skill level is an exogenous
variable. Required T/P (required time per paper) is the time that is needed to produce a paper in
normal quality level.
The other indicator of quality is reserved time. If reserved time is lower than the required
time, which is the time that is needed to produce a paper in normal quality level, then the paper
will be a low-quality one. As the time spent on a paper increases, its quality level increases. The
formulation of quality in the model is seen below.
Quality = Effect of Research Time per Paper * Effect of Skill
A good-quality paper gets more citations than the others. So there is a positive
relationship between quality and number of citations. Additionally, if a paper is a good-quality
one, its obsolete time is longer. Obsolete time is the time that how long a paper stays in the stock
published papers (PubPaper) after being published.
3.1.4. Reputation
Ext Gavg IntC Gavg
Erfextd
on Rep
PerRepChange
Figure 10: Reputation in the model
Reputation of the department is determined by the average number of citations of the
department. The average external citations and the average internal citations of the faculty are
compared with the grand average (world average) values which are obtained in data analysis
part. Since there is a time delay in perceiving any change in reputation, first order information
delay structure is used in modeling this part.
Actual Reputation-Perceived Reputation
Perceived Reputation Change= Reputation Adjustment Time
Reputation is directly affecting the external citations because reputation means being
known by the other academicians. The other academicians prefer to cite from the one whom they
know rather than form anyone On the other hand, reputation does not affect internal citations
since internal citations are coming from the colleagues of the researcher. Reputation has also a
positive effect on the acceptance probability of a paper by a journal.
0.800 0200 || eee
Sige on ave on
0.050 ceed Yd
0.200 ie i ope : :
zB {ao (2 coo mma a AvglntC\P/IntC_Gavg
AvgExtC\P/ExtC_Gavg
Figure 11: External and internal citation effect on reputation
3.1.5. Acceptance Probability
PerRep
Max ActRep
PublishT”
ResinW
NewBomP
PublishR
Figure 12: Acceptance probability in the model
AccPr (Acceptance Probability) is the probability that a paper is accepted by a journal.
PublishT (Publish time) is the time that a paper waits before being published. When PerRep
(Perceived Reputation) is close to MaxActRep (Maximum Actual Reputation) AccPr is high. In
the same manner, when the perceived reputation is low, AccPr is low. AccPr has a negative
effect on PublishT (Publish time). Le., more reputation means more acceptance probability and
more acceptance probability means less waiting time for the paper before being published in a
journal.
Research in Writing
Tabitha Publish time*Effect of Acceptance Pron Publish Time
4. Model Validation
The purpose of model validation is to assure that the model is an acceptable description
of the real system behavior with respect to the dynamic problem (1). Model validation is carried
out in two steps.
4.1. Structure Validity
Structure test is to check whether the structure of a model is a meaningful description of
the real relations that exists in the problem or not. There are two types of structure tests: direct
structure tests and structure-oriented behavior tests (1).
Direct structure tests assess the validity of the model structure by direct comparison with
knowledge about real system structure. Parameter and variable confirmation, dimensional
consistency and extreme condition tests are included in direct structure testing (1). In the model,
all parameters and variables have real life counterparts, there is no dimensional inconsistency in
equations and the model passes the extreme condition tests.
One of the tests in indirect structure testing is extreme-condition test via simulation. In
order to validate the model some extreme conditions are simulated. One of our external input
variables is skill level. The upper extreme for skill level is 100. When we start the simulation
with a skill level of 100, reputation climbs up to the maximum value of 100. It is consistent in
that, if a faculty consists of the most skilled faculty members in the world it becomes the most
reputed one in the world.
Another extreme-condition test is applied to the number of faculty. When there is 1
faculty member, all publication stock levels decrease as expected. On the other hand, when we
start with a faculty of 300 members publication stock levels come to equilibrium at high levels.
Additionally, extreme-condition test is done with the total research time parameter. If faculty
members allocate a very small portion of their time to research (for example 5% of a quarter)
then publication stock levels decrease as expected. On the other hand, if very high portion of
available time is devoted to research, publication stocks reach their equilibrium at high levels.
These entire extreme-condition tests are consistent with the construction of the model.
4.2. Behavior Validity
Behavior pattern tests are designed to measure how accurately the model can reproduce
the major behavior patterns of the real system (1). Real data is not available for our case;
however we can judge the resulting behavior of the system. According to our assumptions, there
should be a balance between the pressures and the actions of the faculty. When the behavior is
examined it is seen that time reserved for a paper reaches its equilibrium after a set of decisions
according to publication and citation pressures. This is kind of seeking a balance between
number of papers published and citations received. This main behavior is consistent with our
assumptions.
5. Output Analysis
5.1. Base Run
As seen in Figure 13, new-born papers and research in writing stocks reach their
equilibrium after oscillation. This is a result of negative feedback loops of the model. Mainly,
publication and citation pressures govern these oscillations. Published papers stock has also a
kind of oscillation before it settles down. Because of the fact that there is not an outflow of
obsolete papers, this stock continues to grow.
In Figure 14 pressure effects can be seen. Publication-pressure increases the total
research time while it decreases the reserved time per paper. In the figure, these opposite effects
can be seen easily. Effects of publication pressure reach equilibrium after oscillations. Effect of
citation pressure on the other hand, keeps increasing throughout the time-horizon.
In Figure 15, behaviors of reserved time per paper and total research time per faculty
can be examined. Reserved time per paper is the decision of the faculty on the average time
devoted to a paper as a result of the pressures. Faculty seeks equilibrium for the reserved time
per paper and it results with a damped oscillation. Total research time per faculty is also the
decision of the faculty in terms of the time devoted to research per faculty member per semester.
Faculty seeks equilibrium for it and it results with a damped oscillation. It is seen that behavior
of these two variables are in the opposite direction. This is as expected because if there is a
publication pressure total research time per faculty will increase; however reserved time per
paper will decrease to be able to publish more papers.
Figure 16 shows the behaviors of average citation per paper, perceived reputation and
quality. As we have mentioned before, when citation increases reputation increases. From the
figure this relation can be seen easily. Quality has an oscillation because it is mainly related to
reserved time per paper. Because of the fact that quality is below 1 it effects citation negatively.
[Pa eubpascr 2 NewBomnP 3 ObscleteP Rosine
h 1000
E 205
F or ae
i r / \ ee
‘ Cie ee m
icon age een eee
\ Us
i pee
fl oa Ww a ‘
F ‘83,
bi 3000
i 00
3
be 200 =
F 3
Fi 0
i 4.
200 3000 10000 15200 700.00
ane 1 Quarters 12.00 25 Mar 2008
Figure 13: Paper stocks in the base model
Page 1
TENPubPressure on ToIResT T-EWPubPressure on Rest TEV CiPressure on Rest
1.30"
08s 2
138
\ \ a
115
058
120
a 1 4 1
1
0.25 Y
10
0.00 50/00 100.00 18000 200.00
4 Quarters 11:34 25 Mar 2008 Salf
Figure 14: Pressure effects in the base model
[FO Resened TP Total Res Time
1 2504
i as
1 sao] ee
i a3 oe
4 Oe cee oe cere eras emcee
i
y 2
1 050
i ed f t {
3.00 20.00 100-00 15000 700.00
Quarters 11:84 26 Mar 2008 Saf
Figure 15: Reserved time per paper and total research time in the base model
hoaeenrers 7 PerRep salty
50
1
2
> 1.00
5.25
‘20:
078
490
200.00
12:00 25 Mar 2008 Saf
Figure 16: Average citation per paper, reputation, and quality in the base model
5.2. Scenario Analysis
5.2.1. No Publication Pressure Effect on Total Research Time
If there is no pressure effect on total research time, published paper stock reaches
equilibrium at a lower level than that it does in the base model. It is expected because faculty
will not be able to increase number of publications as much as that in the base model. The
behavior of the paper stocks can be seen in Figure 17. As seen in Figure 18, reserved time per
paper reaches equilibrium at a lower value compared to the base model. This is expected; faculty
cannot increase total research time and to be able to catch the world average of the publication
performance, faculty should decrease the amount of time devoted to each paper.
[PPT PunPaper 7 NewBomP e Obsolete F Res
E tno
2 195
3 5600
a 450
a
fl 900 \.
2 150,
3 s0a0
a 300
i r
H eon s
2 125
Fi 800 [br
a cE
0.00 60.00 10000 16000 200.00
age Quarters 1207 26 Mer 2008 Sal
Figure 17: Paper stocks in scenario 1
[IP 1 Reserves TP 7 Total Res Time
100
00 50.00 100.00 160.00 ‘200.00
rage 1 Quarters 1207 25 Mar 2008 sal
Figure 18: Reserved time per paper and total research time in scenario I
5.2.2. No Citation Pressure
Everything being equal, if there is no citation pressure, the faculty does not keep track of
the citations received and so does not care about quality. The main effect is on reserved time per
paper and on total research time per faculty which can be seen in Figure 20. As expected,
reserved time per paper reaches equilibrium at a lower level than that in the base model. Besides
total research time’s equilibrium value is lower than its being in the base model because of the
same reason. In Figure 19, it is seen that paper stock values are higher compared to the base
model as a result of devoting less time to each paper in the absence of the citation pressure.
a]
; i \ ae
es = ‘
S al
‘ eo 4. 1
= j
.
800 -
a
4140, ui
Figure 19: Paper stocks in scenario 2
[PT Reserved TP 7 Total Res Tine
q Pua tiiecnusmuammcagmenn nancy IER
Bi 0130)
q 040
Bi 0.30.
100.00 160.00 200.00
foge 1 Quarters 12:14 25 Mar 2008 Saf
Figure 20: Reserved time per paper and total research time in scenario 2
5.2.3. Lower Skill Level
Skill level is an input for the quality, as stated before. In the base model it was 50 (normal
value for the quality). Different scenarios are created with different values of skill level. One of
them is carried out with a skill level of 20. In this case paper stock levels decrease as expected
(Figure 21). Besides, because of the fact that quality of the papers is low, citation pressure
occurs. Compared to the base model reserved time per paper reaches equilibrium at a higher
level (Figure 22). Quality decreases and this decrease effects the citation and reputation
negatively (Figure 23).
[Fr PusParer 2 Newborn 3 Ohoolete P eRean
i 860
2 205 4
3 B00 ee
F si —— -
4
= 5
Sa
a 2
fo ae F
fl 020
2 ‘ts,
5 sana
a 300
Es
i
2
B
FE
5000 “00.00 150.00 200.00
aye t uariers 1228 25 Mer 2006 Sa
Figure 21: Paper stocks in scenario 3
[BO Reserves TP 2: Total Res Time
1
2409
z a8
Lt 1
0.00 100.00 180.00 200.00
Pave 1 Quarters 12:25 25 Mar 2008 $a]
Figure 22: Reserved time per paper and total research time in scenario 3
[Pave owe 2: PerRep 3: Quality
fl
2
EE
450
1
i eae ACRE eerers epeveneeternnencrse scenes anmvserneernnseet tenes eaceeasesarncane
G 025
——
1 3.00 1
2 Ey eee
3 0.15
200.00
jage 1 12:25 26 Mar 2008 §;
Figure 23: Average citation per paper, reputation, and quality in scenario 3
5.2.4. Higher Skill Level
In this case skill level is increased to 80 and as a result paper stock levels increased as
seen in Figure 24. Because of the fact that quality of the papers is high, citation pressure is not
effective. As a result compared to the base model reserved time per paper reaches equilibrium at
a lower level. Parallel to that total research time has a lower equilibrium than that of the base
model. These behaviors can be seen in Figure 25.
In Figure 26, it is seen that quality increases with the high skill level. As a result of the
higher quality compared to the base model, average citation values increase and reputation
increases.
[FFT robraner 2 NewsonP T-Obsolete F Resin
F au) | as
i ae — '
Po i= =>
=
a ch
: he
: ra
: me
; sty
| _
4 ai
: os
: ue
i &
a ae i mn ani
al elses a rres
Figure 24: Paper stocks in scenario 4
DO heaad TF ——
; aa
H 38
4 i
f ig
; ; :
j j ;
; eo
1: 0.50 *
H $8
tn A an TE =a
UE mae ee
Figure 25: Reserved time per paper and total research time in scenario 4
20
Parr Take bay
1
z
3
Paget Quarters 12:36 _26 Mar 2008 Sal
Figure 26: Average citation per paper, reputation, and quality in scenario 4
5.2.5. Lower Initial Reputation and Higher Skill Level
To be able to show the effect of the initial population and the skill level together, these
last two scenarios are created. In our base model we have taken initial reputation as 50. In this
case initial reputation is 20 and the skill level is 80. As seen in Figure 27, because of the high
skill value quality is high, and reputation climbs up together with citation.
[OT ave cer 7 Perhep 3: Quality
1 11.00
2:
3: 1.80
1 8.00
2 35:
3 138
1 5.00 [7
2 20
3: 0.80:
0.00 50.00, 100.00 160.00 200.00
age 1 Quarters 12:45 _ 25 Mar 2008 Saf
Figure 27: Average citation per paper, reputation, and quality in scenario 5
5.2.6. Higher Initial Reputation and Lower Skill Level
In this last scenario, initial reputation is taken as 100 and the skill level is 20. As seen in
Figure 28, because of the low skill value quality is low, and reputation goes down together with
citation.
21
[BP tare cur 2: PerRep 3: Quality
1
2
3
160.00 200.00
12:47 25 Mar 2008 Sal
page 1
Figure 28: Average citation per paper, reputation, and quality in scenario 6
6. Discussion and Conclusion
The conventional wisdom in the field which is that 10% of the journals get 90% of the
citations is a striking signal of a problem in science policy. The evolution of scientometrics,
which uses number of publications and citations as measure, has to change its direction and find
new comprehensive measures in order to make publication again the way of communication
among scientific peers. For such a new comprehensive measure it is needed a systemic analyze
of the situation in order the grasp the roots of the problem. This study is an initial effort of such
an analysis.
The aim of the study which is to examine the behaviors of researchers in response to
dynamics of publication and citation pressures is achieved as a model including researchers in a
department, their publications, citations and the factors such as reputation, quality, pressures on
researchers and their skill levels.
The main decision of the department (accumulated faculties) is the allocation of time to
produce in high quality or low quality papers and this decision creates the dynamics. In the base
run, quality of publication, reputation of the department, publication and citation pressures are
understood as main factors. High skill and much time results in high quality papers which get
more citations. As papers get more citation the reputation increases and increasing reputation
results in more citation. The positive feedback loop between reputation and citation is very
strong but other feedback mechanism balances it. Publication and citation pressures act in
opposite way. While former causes producing more papers in shorter times (low quality); the
latter tries to make high quality papers in longer times (few paper). These opposite effects make
model reach equilibrium after some oscillations between low and high reserved time values.
In the scenario analysis, when citation pressure is removed the paper stocks reach
equilibrium at higher levels with low quality, less-cited papers. Additionally, the system is very
sensitive to the skill level which is modeled as an exogenous factor. And lastly, when the
reputation and skill level analyzed together, it is seen that the skill level is decisive factor. A
skilled department obtains reputation, regardless of the initial level of reputation.
22
As further research, in order to better grasp the decision mechanism of the researchers,
this model can be widened by including the other pressures (such as career, financial...etc). Or
with an agent based approach the interrelations between multiple departments can be analyzed
with a multi-level model. Furthermore, the network structure between people is a necessary
research topic in understanding today’s problem of science. All in all, this study is an initial
effort and will achieve its goal if it can stimulate any further research.
13.
14.
References
Barlas, Y. 1996. Formal Aspects of Model Validity and Validation in System Dynamics.
System Dynamics Review, Vol.12, No.3. pp. 183 - 210.
Furner, Jonathan. Little book, big book: before and after Little science, big science: a review
article, Part I. 2003, Journal of Librarianship and Information Science, pp. 115-125.
Furner, Jonathan. Little book, big book: before and after Little science, big science: a review
article, Part II. 2003, Journal of Librarianship and Information Science, pp. 189-201.
Garfield, Eugene. Citation Indexing for Studying Science. 1970, Nature, pp. 133-138.
Hamilton, David P. Publishing by -and for?- the Numbers. Science. 1990, Vol. 250.
Kiigiik, Bilge, Giiler, Nisa, & Tamgakir, Selin. Measurement Analysis and Evaluation of
Academic Publications and Citations. Istanbul : s.n., 2007.
Margolis, J. Citation Indexing and Evaluation of Scientific Papers. 1967, Science, pp. 1213-
1219.
Menczer, Filippo. Evolution of document networks. 2004, PNAS, pp. 5261-5265.
Moed, Henk F. Citation Analysis in Research Evaluation. Dordrecht : Springer, 2005
. Moed, Henk F, Glanzel, Wolfgang, & Schmoch, Ulrich. Handbook of Quantitative Science
and Technology Research. Dordrecht : Kluwer Academic Publishers, 2004.
. Newman, M E J. Scientific Collaboration Networks. I.Network construction and
fundamental results. Physical Review. 2001, Vol. 64, p. 016131.
. Price, Derek J. de Solla. Ethics of Scientific Publication. 1964, Science, pp. 655-657.
Price, Derek J. de Solla. Networks of Scientific Papers. 1965, Science, pp. 510-515.
—. Scientific Collaboration Networks. II.Shortest paths, weighted networks, and centrality.
Physical Review. 2001, Vol. 64, p. 016132
23
