PAP E R S
Performance assessment in the maternity pathway
in Tuscany region
Sabina Nuti, Anna Bonini, Anna Maria Murante and Milena Vainieri
Scuola Superiore Sant’Anna, Management and Health Laboratory, Pisa, Italy
E-mail: snuti@sssup.it
Summary
The paper describes the performance measurement system of the maternity pathway used in Tuscany by
health care professionals, general managers and regional policy-makers. This system uses 19 indicators
grouped in six dimensions: population’s state of health; compliance with regional guidelines; efficiency
and financial performance; clinical and health assessment; patient satisfaction; and employees’ satisfac-
tion. The results are represented on a spider diagram that summarizes the results on the different
dimensions. The Tuscan performance measurement system of the maternity pathway has been used to
identify best practice within, and their adoption throughout, the Tuscan public health care system.
Introduction
Sixty-one years ago, in 1948, both the World Health
Organization (WHO) and the United Nations (UN)
recognized the importance of maternal infant care. The
WHO consider it as one of their main functions ‘to
promote maternal and child health and welfare’.1 The
UN secured in the Universal Declaration of Human
Rights the obligation to provide ‘special care and assist-
ance’ for mothers and children.2 The primary health care
movement launched at Alma-Ata in 1978 also recognized
the importance of maternal infant care. The topic received
renewed attention in the last decade when it was identified
as one of the goals of the Millennium Development Goals
as an integral part of poverty reduction.3 The WHO report
of 2005 emphasized that: ‘Public health programmes need
to work together so that all families have access to a con-
tinuum of care that extends from pregnancy (and even
before), through childbirth and on into childhood,
instead of the often fragmented services available at
present’.4
In Italy, National Health Plan (NHP) 1998–2001
identified a strategy of developing maternal and infant
care as a strategic pathway, indicating actions required to
achieve the strategic objectives.5 Following the 2000
Ministry of Health’s adoption of the ‘maternal and infant
care objective project’, regions emphasized the importance
of maternal and infant care in their Regional Health Plans
(RHP). Since 1998, the RHPs of Tuscany6 – 8 have sought
to implement the national strategy and achieve national
objectives. Two regional laws (DGR n. 555 and the
DGR n. 784 both of 2004)9,10 stipulated guidelines for
the Tuscan Local Health Authorities (LHAs) for the
maternity pathway, and actions to be implemented, to
renew and develop the regional network of maternity
and infant care. These laws resulted in numerous training
initiatives and organizational changes.
This paper aims to describe the performance measure-
ment system of the maternity pathway (PMSMP) adopted
by Tuscany region from 2004 in order to assess and
monitor its LHAs and Teaching Hospitals (THs).11,12
Conceptual framework
The PMSMP was developed using lessons from the many per-
formance measurement systems that have been developed
over the last 20 years,13– 15 in particular from multidimen-
sional systems already used in health care16– 18 and the
model of the Balanced Scorecard.19 –21 One of the lessons
was the importance of developing the PMSMP in close col-
laboration with health care professionals and managers.22–25
The multidimensional evaluation system for the
maternal infant care pathway
The Tuscan region has a low birth rate, although there has
been a small increase since 1995. In 2005, there were
31,433 deliveries, 98% of which were in a public hospital,
with a birth rate of 8.7 newborns per 1000 inhabitants,
lower than the national one that was 9.5 per 1000 inhabitants.
Sabina Nuti MBA, Professor of Economics and Public Administration,
Director of Management and Health Laboratory, Scuola Superiore
Sant’Anna, Piazza Martiri della Liberta`, 33-56125 Pisa, Italy.
Anna Bonini MS, Management and Health Laboratory, Scuola Superiore
Sant’Anna, Piazza Martiri della Liberta`, 33-56125 Pisa, Italy.
Anna Maria Murante MS, Management and Health Laboratory, Scuola
Superiore Sant’Anna, Piazza Martiri della Liberta`, 33-56125 Pisa, Italy.
Milena Vainieri MS, Management and Health Laboratory, Scuola
Superiore Sant’Anna, Piazza Martiri della Liberta`, 33-56125 Pisa, Italy.
Health Services Management Research 2009; 22: 115–121. DOI: 10.1258/hsmr.2008.008017
The PMSMP consists of six dimensions of assessment
(the letter is used to indicate each dimension):
(a) Population health, such as the infant mortality rate
(A1);
(b) Regional health system, to guarantee equity of access
throughout the region, but also points recommended
by WHO such as Breastfeeding (B6.1), Continuity
of care (B6.2), Humanity of care (B6.5);
(c) Quality, appropriateness, effectiveness and managing
supply to match demand; e.g. Nullipar Terminal
Single Vertex (NTSV) caesarean rate (C7.1) for
women at the first delivery with the minimum of 38
weeks of pregnancy, no twins and in vertex position;
(d) Patient satisfaction, the experience of women who
have given birth on care and help provided through-
out the care pathway (from the prenatal to the post-
natal phase);
(e) Staff satisfaction, results of surveys on the satisfaction
level of staff who deliver care with their working con-
ditions and management by the LHA;
(f) Efficiency and financial performance, such as the
number of deliveries per gynaecologist and number
of deliveries per obstetrician.
In order to display performance by each dimension,
a spider chart was used with assessment on each indicator
classified into the six dimensions, being divided into five
bands with different colours (with scores in parentheses):
(1) Dark green (between 4 and 5), excellent performance
(achieving the target);
(2) Light green (between 3 and 4), good performance;
(3) Yellow (between 2 and 3), average performance and
ample scope for improvement;
(4) Orange (between 1 and 2), poor performance that can
and must be improved;
(5) Red (below 1), failing performance that can and must
be improved urgently.
An LHA with a high score on an indicator for each
dimension will be displayed as close to the centre (dark
green) and one with a low score will be displayed as far
from the centre (red) (Figure 1).
Selection of indicators
Performance on each dimension of the spider diagram is
the sum of scores across a ‘tree’ of indicators (Table 1).
In 2004 indicators were selected from those used in other
systems such as Ontario (Canada), UK and USA.26,27 These
were presented and discussed with health care professionals
and managers throughout the region in meetings, focus
groups, seminars and in a consensus conference. They
were considered in terms of the availability of data and
Figure 1 The target diagram of the Tuscan maternity pathway evaluation
116 Nuti et al.
Health Services Management Research 2009 Volume 22 Number 3
then reviewed by professors of THs in a training pro-
gramme that involved the heads of maternal and infant
care departments. The indicators identified are listed in
Table 1; some are combinations of a number of indicators.
Data collection
The sources of data for the different indicators were as
follows (Table 2):
(a) Population health, regional register of mortality;
(b) Regional health system, the Certificati di Assistenza al
Parto – certificate of birth assistance (CAP) and
women’s experience;
(c) Quality, the CAP;
(d) Patient satisfaction was obtained by telephone survey
of women who had given birth at least 30 days pre-
viously (using the Computer-Assisted Telephone
Interview methodology by the Management and
Health Laboratory in collaboration with the
Statistical Sector of Tuscan Region);
(e) Staff satisfaction, by using the Computer-Assisted
Web Interviewing methodology by the Management
and Health Laboratory;
(f) Efficiency and financial performance, LHAs offices for
statistics.
Performance evaluation
The benchmark for performance on each indicator was
chosen in one of the following two ways:
† Ideally, by using a recognised international or regional
standard, e.g. the WHO standard28 – 30 for the percen-
tage of caesarean births;
† Where no obvious standard existed, the regional
average was used with adjustments where necessary
for different risks between health authorities.
The performance of each health authority on each
indicator was scored by creating regular intervals around
the benchmark.
Table 1 The maternity path indicators
Code of indicator Category Indicators
A1 Population’s health Mortality within the first year of life
B6.1 Capacity to follow regional strategies Percentage of breastfeeding
B6.2 Capacity to follow regional strategies Continuity of care
B6.3 Capacity to follow regional strategies Humanization of care path
B6.4 Capacity to follow regional strategies Equity and access to service
B7 Capacity to follow regional strategies Mobility rate
C7.1 Clinical and health assessment NTSV caesarean rate
C7.2 Clinical and health assessment Rate of induced births
C7.3 Clinical and health assessment Episiotomy rate
C7.4 Clinical and health assessment Percentage of infant with Apgar index ,7 at the fifth minute
C7.5 Clinical and health assessment Underage pregnancy rate
D8.1 External assessment Women satisfaction rate regarding all the pathway
D8.2 External assessment Women satisfaction rate before labour
D8.3 External assessment Women satisfaction rate regarding labour
E1 Internal assessment Internal climate survey response rate
E6 Internal assessment Employee satisfaction rate with the working condition
E7 Internal assessment Employee satisfaction rate with their management
E9 Internal assessment Employee satisfaction rate with training activities
F1 Efficiency and financial performance Resource efficiency
Table 2 Data source by dimension
Dimension Source of data Methodology
Population’s health Administrative regional data Query on regional databases
Capacity to follow regional
strategies
Administrative regional data and patients’
satisfaction and experience
Experiential items coming from the survey and query on
regional databases
Clinical and health
assessment
Administrative regional data Query on regional databases
External assessment Patients’ satisfaction and experience Survey through Computer-Assisted Telephone Interview
technique, Satisfaction items
Internal assessment Staff satisfaction Survey through Computer-Assisted Web Interviewing
technique
Efficiency and financial
performance
Administrative authority data Local Health Authorities and Teaching Hospitals databases
Performance assessment in maternity pathway 117
Health Services Management Research 2009 Volume 22 Number 3
The results of the implementation of the
multidimensional evaluation system for the
maternal and infant care pathway
The results of the PMSMP introduced in 2005, shown in the
spider diagram in Figure 1, suggest that the performance was
mostly good. Tuscan mothers were satisfied with the individ-
ual aspects of care in the maternal and infant care pathway
both in the prenatal and delivery phases. The performance
on the different dimensions was as follows:
(a) Population health, during 2002–2004 period: Tuscan
mortality rate within the first year of life was 2.99 per
1000 new born alive, which was smaller than the
national rate (4.04), while the rate of each LHA had
great variability (from 2.00 to 4.88 per 1000 born
alive); the regional performance is good (on light green
band);
(b) Regional health system: Although across the region
performance on average is good on Breastfeeding
(B6.1), Continuity of care (B6.2) and Humanity
(B6.5), there is variation with scope for improvement
by some LHAs and THs (Figure 2 reports scores for
each LHA). Equity and access to services (B6.4)
obtains the most critical result. About 53% of preg-
nant women participate in the prenatal training
course, but this result decreases to 16% if only
women with low education are considered. A strong
connection between participation and education
level is observed (Figure 3);
(c) Quality: At the regional level all indicators fall into
the yellow band and differences have been observed
among the performances of health authorities.
Figure 4 reports the variability of NTSV caesarean
rate (C7.1) within LHAs and THs;
(d) Patient satisfaction: More than 90% of women stated
that they were completely or very satisfied1 with the
aspects related to all phases of the maternal and
infant path (D8.1), the prenatal phase (D8.2), the
labour phase (D8.3) and the coordination along the
pathway (B6.2). Although satisfaction was very high
across the region, there were statistically significant
differences between LHAs, and hence scope for
improvement by some (Figure 5);
(e) Staff satisfaction: The surveys of 2005 and 2007 were
carried out on a representative sample of employees
in all the LHAs and THs. On a regional level, training
activities and professional working conditions (E9 and
E6) were assessed positively, but the employee satisfac-
tion with their management (E7) obtained an inter-
mediate assessment. The percentage of employees
who work in the maternal and infant sector who
took part in the survey was very low and points out
a higher difficulty to participate. This is confirmed
by comparing the response rate of the rest of the
employees with that of the maternal and infant pro-
fessionals as shown in Figure 6;
(f) Efficiency and financial: The relationship between
human resources engaged within the maternal
infant care pathway and the services delivered
results different among the LHAs. In Tuscany each
gynaecologist follows a mean of 91.83 deliveries
per year (Figure 7). LHAs under the mean were
considered less efficient and they obtained a lower
score.
It is interesting to note that there is a significant cor-
relation between the high level of efficiency (number of
deliveries per gynaecologist) and the good performance
on the clinical dimension (NTSV caesarean rate)
(P, 0.05), such as achieved by the LHA 4 and LHA
5. This proves that resource volume is not related to the
quality of care.
Figure 2 The performance scores of the indicators of B dimensions coming from the survey within
Local Health Authorities (LHAs)
1Women could have chosen among five modalities of answers: completely
satisfied; very satisfied; fairly satisfied; little satisfied; and not satisfied at
all.
118 Nuti et al.
Health Services Management Research 2009 Volume 22 Number 3
Figure 4 The distribution of the Nullipar Terminal Single Vertex (NTSV) caesarean rate among Local
Health Authorities (LHAs) and Teaching Hospitals (THs)
Figure 3 The distribution of the access to prenatal course by the
educational qualification of the mothers in a Local Health Authority
(indicator B6.4)
Figure 5 The mean satisfaction of maternal infant path phases within Local Health
Authorities (LHAs)
Performance assessment in maternity pathway 119
Health Services Management Research 2009 Volume 22 Number 3
Discussions and conclusions
The application of this multidimensional evaluation
system to the maternity pathway has been well accepted
and it is now used in the local budget systems by the
health care professionals and managers in Tuscany. What
has facilitated the introduction of the system was that for
the first time in Tuscany it was possible to integrate data
from the regional information system and field surveys con-
sidering performance in various dimensions, and that they
were presented in benchmarking.
A second point that helped the introduction of the
system was the easy and effective way through which
data are reported and presented visually. The target and
its colours have become a common language among obste-
tricians, physicians, nurses and managers. It also represents
a friendly way through which health care results are com-
municated to the public.
All LHA managers, the regional councillor and all
health care professionals receive a booklet and can explore
the indicators and analyse the relationship between the
results obtained on the web (see www.valutazionesanitatos
cana.sssup.it). Some of the indicators, such as the continu-
ity of care and the NTSV caesarean rate, are also linked to
the incentive system for the general managers of the
LHAs.
The above-mentioned issue and the data public release
had an impact on the quality of the service delivered,
especially in the LHAs who performed badly.31 These
LHAs adopted actions to improve their performance and
to organize training courses involving obstetricians, physi-
cians and nurses working both in the hospital and in
primary care services.
Through PMSMP it has also been possible to identify
and highlight at which level of the regional health system
problems exist. If a particular indicator shows a negative
performance for all the health authorities, it is a general
problem that requires attention at a regional level.
If, however, there is variability between the health autho-
rities, it means that health authorities can improve cons-
idering the best practice.
Health authorities with less favourable results tend to
look at those who perform well in specific areas and adopt
similar more effective and efficient strategies.32
The system represents a rich assessment tool for the
regional council, especially because it is used systemati-
cally. It supports the regional government in setting
Figure 6 The response rate of the climate survey of both the overall and
maternity path
Figure 7 The distribution of the number of deliveries per gynaecologist among
Local Health Authorities (LHAs) and Teaching Hospitals (THs)
120 Nuti et al.
Health Services Management Research 2009 Volume 22 Number 3
health care priorities and allocating funds to assure a high
quality of care.
However, there are still many areas to be investigated
and improved. Although not perfect, the Tuscan PMSMP
gives a real picture of the quality of care of the pathway;
it highlights the strengths and weaknesses of each single
LHA in the different areas. Future research should investi-
gate the main levels through which improved performance
can be realized and the relationship among the various
indicators.
Acknowledgements
We thank all staff of the Management and Health
Laboratory for improving a positive learning environment
despite hard work. Special thanks to Sara Barsanti,
Alessandro Campani, Domenico Cerasuolo, Elisabetta
Felloni, Barbara Lupi, Cristina Renzi and Daniel De Wett.
References
1 Constitution of the World Health Organization, Article 2. Geneva:
World Health Organization, 1948
2 United Nations Universal Declaration of Human Rights. New York,
NY: United Nations, 1948
3 Human Development Report 2003. Millennium Development Goals:
A Compact Among Nations To End Human Poverty. New York, NY:
Oxford University Press for the United Nations Development
Programme, 2004
4 WHO, The World Health Report 2005. Make Every Mother and Child
Count. WHO, 2005
5 Minestero della Salute. National Health Plan (Piano Sanitario
Nazionale) 1998–2001
6 Regional Health Plan of Tuscany (Piano Sanitario Regionale
Toscana) 1998–2001
7 Regional Health Plan of Tuscany (Piano Sanitario Regionale
Toscana) 2002–2004
8 Regional Health Plan of Tuscany (Piano Sanitario Regionale
Toscana) 2005–2007
9 Delibera Giunta Regionale Toscana n. 555 of 2004
10 Delibera Giunta Regionale Toscana n. 784 of 2004
11 Nuti S, Il sistema di valutazione delle aziende sanitarie toscane, in
S. Baraldi (a cura di) Balanced Scorecard nelle aziende sanitarie.
Milano: McGraw Hill, 2005
12 Laboratorio Management e Sanita`, Report 2006. Il sistema di valuta-
zione della performance della sanita` toscana. Edizioni ETS, 2007
13 Lewis J. The EFQM Excellence Model. London: Woodward Lewis,
1999
14 Mayne J, Zapico Goni E. Effective Performance Monitoring:
A Necessary Condition for Public Sector Reform Monitoring
Performance in the Public Sector. New Jersey: Transaction Publishers,
1997
15 Kaplan RS, Norton DP. Putting the balance scorecard to work.
Har Bus Rev 1993;71:134–47
16 Canadian Institute for Health Information, Government Hospital
Association and the University of Toronto. Hospital Report ‘99:
A Balanced Scorecard for Ontario Acute Care Hospitals. Toronto:
Ontario Hospital Association, 1999
17 Pink GH, McKillop I, Schraa EG, Preyra C, Montgomery C, Baker
R. Creating a balanced scorecard for hospital system. Health Care
Finan 2001;27:1–20
18 Canadian Institute for Health Information. The Ontario Health
System Scorecard. Health Results Team for Information Management.
Queen’s Printer for Ontario, 2006
19 Kaplan RS, Norton DP. The Balanced Scorecard: Translating Strategy
into Action. Cambridge MA: Harvard Business School Press, 1996
20 Kaplan RS, Norton DP. Using the balanced scorecard as a strategic
management system Har Bus Rev 1996;74:75–85
21 Kaplan RS, Norton DP. Linking the balanced scorecard to strategy.
California Manag Rev 1996;4:53–79
22 Pursglove J, Simpson M. A balanced scorecard for university
research. In: Neely A ed. Performance Measurement: Past Present
and Future. Cransfield: Cransfield School of Management, 2000
23 Aidemark LG. The meaning of balanced scorecards in the health-
care organization. Financ Account Manag 2001;17:23–40
24 Abernethy MA, Stoelwinder JU. The role of professional control
in the management of complex organizations. Account Organ Soc
1995;20:1–17
25 Jones CS, Dewing IP. The attitudes of NHS clinicians and medical
managers towards changes in accounting controls. Financ Account
Manag 1997;13:261–80
26 Health Canada, Health Policy and Communication. Canada Health
Act Division, Canada Health Act, Annual Report, 2001
27 Sutter Women’s & Children’s Services. First Pregnancy and Delivery.
Clinical Iniziative Resources. California: Sutter Health, 2001
28 WHO. Appropriate technology for birth. Lancet 1985;2:436–7
29 WHO Department of Reproductive Health and Research. Care in
Normal Birth: A Practical Guide. Report of a Technical Working
Group. WHO/FRH/MSM/96.24. Geneva: WHO, 1996
30 Department of Reproductive Health and Research. WHO Antenatal
Care Randomized Trial: Manual for the Implementation of the New
Model. WHO/RHR/01.30. Geneva: WHO, 2002
31 Bevan G, Hood C. What’s Measured is What Matters: Targets and
Gaming in the English Public Health Care System. In: Public
Administration, Vol. 84, 2006a
32 Mc Nair CJ, CMA, Leifried KHJ. Benchmarking: A Tool for
Continuous Improvement. New York USA: John Wiley & Sons, 1992
Performance assessment in maternity pathway 121
Health Services Management Research 2009 Volume 22 Number 3