Transformative quality of doctoral education: the way new standards are negotiated. System
dynamics approach.
Birute Mikulskiene, Daiva Mazrimiene
Institute of Management, Faculty of Politics and Management, Mykolas Romeris University,
Valakupiy g. 5, LT-10101 Vilnius, Lithuania
Abstract
Quality is a concept that leads every discussion about public policy of higher education (HE).
Typically, state's external regulation of the quality of doctoral education is supported by internal
self regulation, where a certain level of the quality of doctoral education is ensured by long lasting
internal “common” practice. For example, the process of doctoral degree awarding (DDA) is an
instance where the state regulates requirements for the process and personnel involved.
The goal of this study is to analyse the practice of doctoral degree awarding in Lithuania
(selection of DDA members and the DDA process) and determine actual factors that stimulate
transformation of the quality of doctoral education. The research is based on the qualitative
approach using semi structural interviews. Informants for interviews were selected from the list of
DDA committees that operated in Lithuania in 2010-2011. On the basis of the qualitative research
data, we have constructed an explanatory causal loop diagram explaining the nature of the
transformation of the quality that is a result of negotiation between academia and state regulators.
Dynamic hypotheses are based on threefold causal loops: adoption of new state regulation,
implementation of state regulation and formation of new tolerance to quality.
Keywords: doctoral education, quality, system dynamics.
Introduction
Quality is a concept that leads every discussion about public policy of higher education (HE).
However, uncertainties of describing the quality content leave policy makers in an unfavourable
position as they draft quality regulations. The quality of doctoral education is an even more
complicated issue of HE. Whereas quality dimensions describing training aspects of tertiary
education are more or less explicitly defined (staff performance, curricular, student performance,
financing, management and etc.), the quality of research conducted by doctoral students comprises a
number of more complicated issues to be defined. In practice, complications of describing research
quality issues are tackled by integrating state's external regulation of the quality of doctoral
education with internal self regulation, where a certain level of the quality of doctoral education is
ensured by long lasting intemal “common” practice based on peer perception of what a “quality”
research is. For example, the process of doctoral degree awarding (DDA) is an instance where the
state regulates requirements for the process and personnel involved. The qualitative criteria to
assess a thesis of a doctoral student are reserved for peer evaluation typical in a self regulated
practice. A certain level of self regulated quality may change from time to time in respect to the
requirements set by external regulation and peer tolerance of peers towards quality standards.
However, there are limited empirical evidences about effectiveness of self regulation and its impact
on the quality. Even more interesting is to know how extemal regulation applied in concordance
with self regulation determines new standards and encourages adaptation of strategic behaviour of
scientists within the system.
The goal of this study is to analyse the practice of doctoral degree awarding (selection of
DDA members and the DDA process) and determine factors that could lead to transformation of the
quality of doctoral education. Qualitative research methodology using semi structural interviews
was used to collect necessary data. A system dynamics approach has been used with the purpose to
interpret the non linear evolution of quality changes when external regulation is triggered by self
regulated practice. Empirical evidences let us shed some light on the dark side of the DDA process
and associate the process with perceptions of self regulation as guidelines for policy making when
new regulation is being adopted.
Relevance of System Dynamics for quality of doctoral education.
System dynamics is becoming more and more attractive to explain different HE aspects.
System dynamics modelling is used in planning, funding, quality management and monitoring of
higher education (Kennedy, 1998) and educational policy (Kennedy,2008). Moreover, a range of
decision support tools such as UNIGAME have been proposed for strategic University management
based on system dynamics (Barlas et al, 2000).
System dynamics approach suggests a comprehensive and actionable explanation of any type
of phenomena due to its main characteristics. First of all, system dynamics as a non linear approach
enables researchers to tackle problems of hidden nature as the quality of doctoral education is. The
second advantage is the possibility to incorporate driving forces into realisation of the system that
takes the whole system to a new plausible position after it is triggered.
System dynamics approach fits very well to describe the dynamical nature of quality in
doctoral education and tackle the transformative grounds due to:
e complexity of dimensions of quality;
e dynamics of quality development;
e diversity of perception of quality.
Doctoral degree awarding in Lithuania: overview
Doctoral graduates have to develop different skills that are useful beyond one's academic
carrier. The most accurate definition of skills that need to be developed and focused on by doctoral
education quality management is proposed by Bogle and co-authors: “Doctoral researchers are the
drivers of their professional development while being immersed in a research rich environment
where boundaries to other research fields are highly permeable and in which connections to the
external world have a global outlook and link to other sectors of society.” (Bogle et al, 2011).
Lithuanian HE system provides doctoral education in doctoral schools via structural doctoral
programmes. The state regulation have continuously manifested in decrees on regulation power
during the last 20 years and doctoral schools have been gradually empowered to keep an
appropriate quality level of doctoral education and still retain the freedom from state regulation in
ensuring the quality of research output. From this point of view, state regulation is limited to setting
the initial conditions for procedures and control of the requirements for people involved in doctoral
studies. To sum up, we can state that the quality of doctoral education has become a matter of self
regulation where a doctoral school has the right to validate the doctoral degree.
Doctoral degree awarding process. Doctoral training process is finalised by a thesis which
is supposed to be original. Originality and contribution to knowledge is assessed during a defending
procedure with peer review as the main methodology. Defence of doctoral thesis in a public sitting,
that is equivalent for doctoral examination, is also used in other EU countries (UK, Australia). A
doctoral degree is awarded when the awarding committee makes a decision. The awarding
committee's meeting is the last stage of doctoral studies when the doctoral student’s research
abilities — in the form of a thesis — is presented and evaluated. The sitting to present a thesis has a
mission to analyse the research conducted by the student with a possibility to thoroughly discuss the
research topic and make reflections by external peer evaluators.
Previous research. Previously, we have analysed the practice of DDA process in Lithuania
during two selected years (2010-2011) and reconstructed the structure of DDA committees in terms
of social network analysis (Mikulskiene et al, 2013). DDA committees have a duty to ensure the
quality of doctoral research at the final stage of doctoral education. Since the state regulates
requirements for committee members, we selected the issue of regulation for our investigation and
hypothesised that the freedom to select any member whose competence fits the external
requirements is a safeguarding managerial instrument to attain quality. We investigated networks
composed of prominent researchers, who are officially nominated by the doctoral school and
represent the doctoral school's appreciable level of quality. This quality level is a subject of self-
regulation, when committees make a decision according their understanding of the quality of
research. The findings raised significant doubts about practical selection of committee members and
the DDA process. First of all, some members of DDA committees took part in sittings ten times (up
to thirty per year) during the sample period. Another finding concerns closeness of the network. The
networks of DDA committees have demonstrated stability in their composition and some isolation
when the most popular members are not recognised by other doctoral schools in the field. That
makes us wonder why the level of the quality of one doctoral school is not acceptable for another. It
seems that some tolerance towards the quality in one particular committee is reiterating itself in
another committee with the similar composition. If we agree that the network analysis of DDA
committee members affords spotting self-regulation, we can draw a conclusion that the mode of
self-regulation is less competitive and has tendencies to work in a more isolated environment
seeking to avoid outer influences, frequently critical to peer review and biased in favour of colleges
and close partners. To understand the origin of transformation of the quality and the source of new
tolerance produced by self regulation, the system dynamics approach was applied as key
instrument.
Methodology
The research is based on the qualitative approach using semi structural interviews. Interviewees
have been selected from the list of awarding committees that operated in Lithuania in 2010-2011.
This period was selected intentionally seeking to collect data different from that used in previous
study (Mikulskiene et al, 2013). Those competing data could be used for triangulation. Five
interviews were conducted with committee members representing five research fields: humanities,
social science, biotechnology science, technology science and physical science. Each interviewee
represents a different role in the process of doctoral degree awarding: chairman, member or
opponent.
Demographic characteristics of the interviewees are as follows:
e affiliations: all informants are employed in Lithuania's HE institutions;
e age: 48-76 years old;
e sex: 60 per cent are males;
e occupation: most of interviewees are recruited for both research and administrative activities
(dean, vice-dean, head of department);
e recognition: all respondents are active researchers, some of them are internationally
recognisable;
e experience: all respondents were invited to DDA committees a moderate number of times
(3-7).
The semi structural questionnaire was made of open ended questions prepared in advance.
Seeking to build mutual trust between the interviewer and interviewees and to encourage
respondents to speak, questions were asked in order of interviewee preferences.
Semi structural interviews included 13 questions grouped into four topics of interest:
e DDA committee formation: selection of members.
e DDA committee performance.
e The role of the doctoral supervisor.
e Suggestions for the development and improvement of the DDA process.
Findings: DDA committees trapped between external regulation and self-regulation
Impact of state regulation. All respondents highlighted that formation of DDA committees
is based primarily on valid legislation. DDA committees of 5 members with two opponents who
have a role of external reviewers are typical. All members are invited from prominent researchers
around the country with rare exceptions for foreign researchers.
The practice to have both the student supervisor and members who share co-authorship with
the student in the committee used to be typical earlier but the latest legislation has changed the
situation. All respondents stated that they approve such provisions of the new legislation.
Supervisor's role in selection of DDA members. Decisions to invite toa DDA committee a
member from a University other than that the PhD thesis has been prepared in depends entirely on
legal regulation: “primarily, we enrol members from our university, and only then, we refer to other
institutions as legal provisions require (A)”. The main responsibility of selecting appropriate
members to a DDA committee is delegated to the doctoral school and its supervising committee.
However in practice, the supervisor of the student or, in limited circumstances, the student himself
selects the candidates to the committee and proposes their candidacy for approval: ,, the list of
candidates is usually drafted by the supervisor; he is the most interested party" (D). These findings
are not unexpected since scholars have already reported such type of practice in earlier publications.
(RuZeviéius ir kt., 2008, p. 109).
Preferable competencies of DDA members. The list of main competences acknowledged
for DDA members includes scientific excellence, ,,scientists are selected with good publication
records on the relevant topic” (B), activeness and academic recognition, ,,...1 appreciate hard
working scientists and those who publish a lot... “ (E), ,,...We are searching for those who have
prominent achievements..." (E). The main criterion, apart from research competence, is
specialisation or the requirement to invite those researchers whose research interests are similar to
that of the theses ,,with experience...publications in the field “(D).
Self regulation: the reason to refuse an invitation to take part in DDA committees. To the
question, under what circumstances scientists would refuse to take part in a DDA committee,
representatives of research institutes mentioned that the only reason to refuse participation is the
quality of the thesis to be defended (,,in most cases I accept the invitation when the thesis (the
quality of the thesis) is acceptable for me. If thesis is week from the scientific point of view, I
refuse" (A)). The reasons to refuse are also associated with the scientific reputation ,,..what is the
point to accept an invitation if there is a risk to tarnish my reputation? Lithuania is a small country,
everybody knows each other“ (A). Other respondents feel confused to refuse participation: ,,when I
accept an invitation, I usually know nothing about the quality of the thesis, and if the thesis is
week, the refusal to participate is a bad manner... “ (E).
Self regulation: frequent participation. Al respondents have confirmed that some scientists
participated much more often than others: ,,actually, some scientists take part in every DDA
committee while others are never enrolled" (C). Trying to find out why some scientists participate
ten times while others only once, we compiled a list of explanations. First, all respondents refer to
the size of the country: “Lithuania is small country, so scientists know each other” (D), ,,we know
all scientists working on the topic and on the bases of this knowledge we select an appropriate
member, usually the same " (A), ,,...the supervisor is very well aware about the informal data base
of appropriate candidates, no specific list is needed...“(C). The second reason is scientific
competencies and academic achievements (,,we are simply aware that these people will have their
motivate opinion“...(C), as “they publish a lot and periodically, their knowledge is up to date” (C).
The third and the most serious explanation of frequent participation are based on personal
behaviour during the DDA process and critical style of presentation. ,,Comments of reviewers, you
will always find a point for critique and you as an opponent can number plenty of them, but the
form you present them.... It could differ from one person to another..." (D),,, your comment could
be offending..." (D). People who present reviews too explicitly and in a too detailed way are
avoided. Ill-disposed people are also eliminated. ,. Nobody wants to deal with ill-disposed people”
(A).
Despite that, all respondents acknowledged the fact that many scientists participated in DDA
committees plenty of times, they also highlighted that this activity is labour intense and this fact
limited the possibilities of meaningful participation ...“ it is not possible to participate in every
committee” (C).
Finally, when DDA members are selected from among the scientists that are qualified as
prominent, the priority is given to the scientists affiliated with the university the thesis has been
prepared in, and other members are invited just to follow requirements of external regulation. The
supervisor of the doctoral student takes an initiative to search for appropriate members and the
doctoral school accepts his/she proposals with rare exceptions. Among different reason of refusing
to take part in DDA committees is the risk, that the quality of the thesis may be too poor and
participation in such defence meeting is perceived as some how damaging the scientist's reputation.
Results: transformative quality of doctoral recognition
Presumably, a certain quality level of applicable DDA committee proceedings is a result of an
encounter between two opposing parties: a policy maker as regulator with strict willingness to
extend state regulation upon the quality and an academia with a long lasting practice of operation at
a certain quality level. The balance of extemal state regulation and self regulation produces the
quality level that is agreed by both parties (Figure 1).
extemal state
regulation
¥
agreed quality for
doctoral educational
process
self regulation to operate at
the certain quality level of
doctoral education
Fig. 1. Forces that effect transformation of quality.
Thus, an assumption that an agreed quality level is impossible to be steadily defined may be
made. On the contrary, it could change once a new trigger in the doctoral education system is
introduced. The agreed quality level becomes a mater of negotiation that depends entirely on
particular aspects of the quality that are regulated.
external state
regulation
2
+ requirements for doctoral
degree awarding committee
agreed quality for members
doctoral +
educational process Pi
‘i moot for
regulation
self regulation to operate at
the certain quality level of
doctoral education
Fig. 2. Dynamic hypothesis for quality agreement: the causal loop for adoption of new state
regulation.
external state
regulation
+ requirements for doctoral
degree awarding committee
agreed quality for members
doctoral
I
educational process
selection of DDA
Ng, maps Z memebers
regulation
self regulation to operate at +
the certain quality level of
doctoral education
Fig. 3. Dynamic hypothesis for quality agreement: the causal loop for implementation of state
regulation.
In the analysed case, the regulated aspects are the requirements for DDA committee members.
Once an agreement about new requirements is reached, new state regulation that has a potential to
shape the agreed quality is adopted (Figure 2). When the external state regulation stipulates only the
requirements for committee members and fails to discuss the remaining issues of the DDA process,
silent issues become a subject of self regulation once the requirements are followed.
Implementation of external regulation together with unregulated issues constitutes DDA practice.
The cycle maintains a balance between state regulation and self regulation within an individual
university and corresponds to a hypothetic level of quality. The transformative impact of self
regulation on the quality exposes a delay as the development of repeatable practice takes several
doctoral education cycles (Figure 3).
The causal loop attached to the loop of implementation of state regulation explains the way
new tolerance towards quality perception is created (Figure 4). The tolerance to the new quality
level could change both directions — increase or decrease the quality. Whatever the course of
tolerance is created, it has a direct impact on the agreed quality transformation via pressure on the
state regulation. Let us now look closer at the causal loop of the new tolerance to quality. Once the
requirements for DDA members are known, the selection procedure can be started. The responses
of the interviewees allow an assumption that invitation of a particular candidate is based on the
professional recognition in accordance with legislation.
non accepted invitation
to participate in DDA
external state: invitation based on
professional recognition
+ accepted invitation to
participate in DDA
+ requirements for doctoral Commies
‘ degree awarding committee +
agreed quality for oe embers ll invited by expectation \
doctoral educational selection of DDA to be non critical
process memebers | }
negotiating for a } willingness to be
regulation | invited repeatedly
new tolerance to
self regualtion to operat at quality ,
the certain quality level of “___ expression of
doctoral education expected behaviour
Fig. 4. Dynamic hypothesis for quality agreement: the causal loop of new tolerance to quality.
The causal loop splits into three branches: one of them corresponds to priorities of the
selector, another two represent a behavioural response of the inviter. According to interviewees,
only candidates who are believed to be more polite and less critical will be selected from the corpus
of suitable candidates. If the choice comes up to expectations, the particular member will be invited
repeatedly in the future. Potential DDA members face a dilemma. It is an honour to be invited as the
invitation means recognition of one's scientific achievements and demonstrates confidence in the
research excellence. On the other hand, the invitation conceals an obligation to behave in a
“silently” agreed manner. The willingness to be invited repeatedly is a motive to demonstrate
competence in an appropriate way. It seems that a critical review of the thesis has little chance to
show up and is not acceptable since when the member is about to express a critical opinion, he/she
would prefer not to participate in a DDA committee to avoid public statement of his/her position
during the committee meeting.
Conclusions
The qualitative research supports the quantitative results of the social network analysis. Particular
members participate more than 30 times per year and make isolated groups. The fact that such
groups have their own star like network lets us make a preliminary conclusion, that the self
regulation mode could be too week to maintain strict quality standards. Week state regulation
makes a tolerant environment to negotiate new standards. Rationality of interviews allows a
hypothesis that a possibility to create a new tolerance towards quality demonstrates a vague ability
of self regulation to safeguard the quality as the willingness to be invited repeatedly motivates
scientists to adjust their behaviour to the expectations of inviters.
On the basis of the qualitative research data, an explanatory causal loop diagram that explains
the transformative nature of the quality has been drafted. The agreed quality becomes a product of
negotiation between an academia and state regulators. The dynamic hypothesis is based on threefold
causal loops: a causal loop for adoption of new state regulation, a causal loop for implementation of
state regulation and a causal loop of new tolerance to quality.
The findings about the transformative quality implicate tendencies for the policy regulation to
follow - to create such regulatory measures that stimulate operation of a healthy self regulation
mode and to stop ill-transformation. The findings could also be generalized to determine what
impact the presence or absence of state regulation might have on the system and what the starting
negotiable point is.
References
1. Barlas, Y., Diker V.G., 2000. A Dynamic Simulation Game (UNIGAME) for Strategic
University Management. Simulation Gaming 31, p. 331-358.
2. Bogle, D, Dron, M, Eggermont, J, and van Henten, J W. 2011. Doctoral Degrees Beyond
2010: Training Talented Researchers for Society. Procedia. Social and Behavioural
Science, 13, 35-49.
3. Kennedy, M. (1998). Some Issues in System Dynamics Model Building to Support
Quality
4. Monitoring in Higher Education, Proceedings of the 16th System Dynamics Conference,
Quebec City, Canada.
5. Kennedy, M. (2000). Towards a Taxonomy of System Dynamics Models of Higher
Education, Proceedings of the 18th International Conference of the System Dynamics
Society (2000) 6-10 August 2000 Bergen, Norway
6. Kennedy, M. (2008) A Taxonomy of System Dynamics Models of Educational Policy
Issues, Conference Proceedings, The 2008 International Conference of the System
Dynamics Society, July 20 — 24, 2008, Athens, Greece.
7. Mikulskiene B., Mazrimiene D. 2013. The quality of doctoral degree awarding:
perspective of self-regulation mode. KIM2013 Knowledge and _ Information
Management Conference, 4 - 5 JUNE 2013.
8. RuzZevitius J. ir kt. Kokybés vadybos taikymo auk&tosiose mokyklose izvalgos // Vie8oji
politika ir administravimas. — Vilnius: Mykolo Romerio universiteto Leidybos centras,
2008, Nr. 24, p. 99-113. — ISSN 1648-2603 .
