Opetus- ja kulttuuriministeriö Ministry of Education and Culture E ffectiveness indicators to strengthen the know ledge base for cultural policy Effectiveness indicators to strengthen the knowledge base for cultural policy Publications of the Ministry of Education and Culture 2011:16 Effectiveness indicators to strengthen the knowledge base for cultural policy Publications of the Ministry of Education and Culture 2011:16 Ministry of Education and Culture • Department for Cultural, Sport and Youth Policy • 2011 Opetus- ja kulttuuriministeriö • Kulttuuri-, liikunta- ja nuorisopolitiikan osasto • 2011 Ministry of Education and Culture Department for Cultural, Sport and Youth Policy P.O. Box 29, FIN-00023 Government Finland http://www.minedu.fi http://www.minedu.fi/OPM/Julkaisut Layout: Erja Kankala, Ministry of Education and Culture Cover: Michael Travers/ rodeo.fi ISBN 978-952-263-037-7 (Online) ISSN-L 1799-0343 ISSN 1799-0351 (Online) Publications of the Ministry of Education and Culture 2011:16 3 Contents Foreword 4 1 Development of effectiveness assessment in central government 6 2 Assessment of effectiveness in cultural policy 10 3 Areas of cultural policy effectiveness goals and the indicators corresponding to them 31 4 Development needs and proposals concerning cultural policy indicators and other strategic information provision 41 Appendix: Cultural policy indicators 55 Sources 69 4 Foreword Finnish cultural policy has been actively developed in recent years. In 2009, the Ministry of Education and Culture drew up its own strategy for cultural policy that extends to 2020, while a government report on the future of culture was completed in 2010. These two documents discuss the changes affecting the cultural policy operating environment, set development targets and propose concrete measures for achieving them. However, simply setting targets and measures is not in itself enough. These days, the implementation of measures within central government needs to be monitored to determine whether the measures have achieved the targets set and whether the targets have been socio-politically relevant. As part of its strategic development work, the Ministry of Education and Culture carried out a cultural policy indicator project. The project was conducted jointly with Statistics Finland and Cupore, the Finnish Foundation for Cultural Policy Research. According to the assignment, the cultural policy indicator development project aimed to: 1) identify and define the key areas of cultural policy effectiveness indicators needed within the cultural administration supervised by the Ministry of Education and Culture, 2) review the need for information on the quantitative and qualitative indicators related to these, 3) collect, where possible, the required indicator data concerning the areas of effectiveness indicators or initiate action to acquire any missing data, 4) develop the statistics and information production system for the culture sector, based on the needs mentioned above, or recommend development measures, as well as 5) draw up a report on cultural policy indicators, describing the key observations and results related to the project’s objectives. The resulting report examines the need to create indicators on the societal effectiveness of cultural policy and the opportunities to create such indicators from the perspective of strengthening the knowledge base for cultural policy. It is very challenging to determine the sector’s societal effectiveness due to the nature of artistic and cultural activities. Cultural activities are creative, active and dynamic and their impact is related to human experiences and interpretations. Moreover, the causal relationships of impacts are difficult to pinpoint and may not appear until quite some time has passed. It can be difficult to distinguish the effectiveness of measures taken by cultural policy actors (such as the Ministry) from the cultural impacts resulting from other actions. They have mutual connections, but cultural impacts may also come about as a result of measures taken by administrative branches other than that of cultural policy. Correspondingly, cultural policy may have impacts other than purely cultural ones. In the field of cultural policy, statistical data is best available with respect to the numbers of actors and participants involved in cultural offerings and the spheres of art and culture. Information is also available on participation in cultural education. Government support for art and culture is known, as is the share of GDP that is accounted for by the cultural and copyright sectors. In recent years, attempts have been made to obtain more information about the cultural economy in terms of consumption, imports and exports. These were some of the aspects that were designated as indicators describing the effectiveness of cultural policy in 5 this project. Not all of them are effectiveness indicators as such, but they still give a picture of the societal effectiveness of cultural policy as a part of the whole group of indicators. The project also identified many issues that are important for demonstrating the effectiveness of cultural policy, but about which there is still too little systematic or reliable information for these issues to be called indicators. Such issues include cultural diversity, culturally sustainable development, intercultural dialogue, participation in culture and the well-being benefits of culture. New challenges include the use, consumption and production of culture online. It is particularly demanding to assess and verify the qualitative effectiveness of cultural policy measures, as well as to evaluate the impacts of culture and cultural policy when they involve many administrative branches. International projects aiming to develop alternative indicators to GDP are interesting in terms of demonstrating the impact of culture. At some point, the culture sector also hopes to get a composite indicator or index that can be used to prove the value or significance of culture to the nation’s physical and mental wealth. The report contains development proposals for obtaining information on these issues. For this to be possible, improvements are needed in academic research, sectoral research and the collection of statistics on cultural policy. The Ministry of Education and Culture must boost its strategic role as a developer in the sector and the part it plays in management through knowledge, in addition to engaging in cooperation with various information producers and users. This report only forms a foundation for longer-term work aimed at strengthening our understanding of cultural policy indicators and related issues. Much remains to be done, although unfortunately there are insufficient resources for carrying out everything deemed necessary. The savings and productivity programmes of central government in Finland will make it more difficult to achieve the objectives. Consequently, the implementation of the development measures included in this report has only just begun. This report includes a review of the prerequisites for producing indicators. It also contains a description of the needs to develop indicators. Some of the needs have been formulated into proposals for improvement. Moreover, the report includes a list of indicators that the Ministry of Education and Culture aims to use based on the information currently available. It is possible that not enough information will be available on all of the indicators. The sources of information vary depending on the indicator. Various statistics provided by Statistics Finland are the main sources, while others include organisations in the field of art and culture, government agencies in the sector, the Ministry of Education and Culture, as well as other ministries. Cupore, the Finnish Foundation for Cultural Policy Research, plays an important role in information production, and the Foundation maintains a research institute of the same name, as well as undertakes academic research into cultural policy, which in Finland is conducted especially at the University of Jyväskylä. This report does not deal with the broader question of how indicators can be used to steer cultural policy. The basic principle is that indicators cannot be used in a more demanding sense as a tool for steering cultural policy until the required knowledge base is solid enough and the indicators themselves are sufficiently versatile and encompass cultural policy operations as comprehensively as possible. This report first describes the project’s background in terms of the demands set by the planning and monitoring systems used for central government finances, and the development needs in the field of cultural policy. This is followed by a description of the starting points and methodology of the development of cultural policy indicators, and a discussion of the development work and relevant research that has already been carried out in Finland and abroad. A presentation is then given of the areas of cultural policy effectiveness indicators determined on the basis of an analysis of cultural policy goals, and the indicators defined for each area. The report ends with conclusions and development proposals related to the range of indicators that have been produced and the working group’s assignment. A detailed list and descriptions of the indicators are provided as an appendix. 6 1.1 Management by knowledge and redefinition of performance Political decision-making and administrative operations require a comprehensive knowledge base of social phenomena and development trends. Knowledge lays the foundation for political guidelines aimed at improving operations in the sphere of social policy as a whole and in its different sectors. Knowledge is not only needed for steering operations, making decisions and implementing measures, but also for evaluating the impacts of policies, decisions and measures. As the global operating environment and society become more complex and the factors affecting development more numerous, the challenges related to management by knowledge increase. The mere existence of knowledge is not enough: one must also be capable of summarising, interpreting and utilising knowledge in policy-making and administration. In addition, it is necessary to critically evaluate the usability of the available knowledge and to identify what is relevant from the ever-increasing mass of information. The Finnish central government has used performance guidance since the 1990s. The central government reform project (2002) resulted in a recommendation to make performance guidance more effective and enhance accountability and performance responsibility. The fundamental maxim of performance guidance was defined as ‘societal 1 Development of effectiveness assessment in central government effectiveness’, which means results related to broad- based socio-political objectives. The reform project’s recommendations were made more concrete by the Working Group on the Final Central Government Accounts (Parempaan tilivelvollisuuteen [Towards better Accountability] (2003)). The steering system for central government finances and operations underwent considerable development after the Budget Act (1216/2003) entered into force at the beginning of 2004. The goal was to develop the final accounts as a whole to promote accountability and decision- making. The Report on the Final Central Government Accounts was formed into a new report on the management of central government finances and submitted by the Government to Parliament. It includes appropriate and sufficient information about adherence to the budget, government revenues and expenditure, the government’s financial position, as well as performance, with the emphasis on information considered to be key in terms of the societal effectiveness of government activities (Indikaattorit [Indicators]... 2005, p. 25). Reporting should be undertaken in as clear and illustrative a manner as possible using both quantitative indicators and qualitative assessments for support. The reform of performance guidance was based on a systematic effort to move the focus of political decision-making from inputs to outputs (Joustie 2009, p. 21). Indicators emphasising effectiveness data, which are already used when setting targets for operations, are key tools in this effort. 7 Output and quality management - services and public commodities - service ability and quality Operational performance - economy - productivity - profitability - cost-effectiveness Human resources development Effectiveness Pe rfo rm an ce g uid an ce Accountability Societal effectiveness How operations and the economy have influenced societal effectiveness Outputs (that can be influenced through management) Societal effectiveness goals Operational performance targets Figure 1. Performance prism indicating the mutual links between performance guidance and accountability, as well as between objectives and outputs. The reforms to performance guidance and accountability have also been linked to programme guidance, which has become increasingly common in recent years. The objective is to get the entire government to adopt the broad-based societal objectives defined in the Government Programme and in Government Policy Programmes instead of each administrative branch acting in line with its own interests and setting its own objectives (cf. Indikaattorit [Indicators]... 2005, p. 30). Administrative branches are bound by the effectiveness goals set by the Government and Parliament, while the effectiveness goals defined by ministries aim to specify and complement the objectives set at the higher level and to make them more concrete (ibid. p. 34). The criteria used to set performance objectives and to report on their success were updated in connection with the reform of performance guidance and accountability. The concepts of societal effectiveness, economic efficiency, productivity and efficiency were brought together under a logical concept and introduced as a performance prism (Figure 1). The performance prism will be used as one of the key frameworks to harmonise government guidance and reporting (Indikaattorit.[Indicators].. 2005, p. 31). Performance is considered to consist of two elements: societal effectiveness and outputs. Since societal effectiveness is the ultimate result and objective of general government activities, it is placed at the top of the prism. Societal effectiveness goals are used to define the operational performance targets of different ministries and, most importantly, the government agencies and departments subject to them. In the philosophy of the performance prism, societal effectiveness goals describe the state of society that policies aim at. This usually involves a change in the circumstances of society or its members. Societal effectiveness refers to the effects that the activities have on entities and societal objectives that aim to satisfy the needs of various social interest groups, ensure peace and stability in society, as well as to safeguard economic growth and well-being (Valtion tilinpäätösuudistuksen periaatteet [Principles for reforming the final central government accounts] 2003, p. 61). Societal effectiveness indicates the degree to which the targets set in social policy have been achieved, that is, how well social policy has succeeded on the whole. In practice, societal effectiveness is the sum of the outputs of numerous players and ultimately results from cross-administrative cooperation. (Ministeriöiden kustannuslaskennan kehittäminen [Developing cost accounting of the ministries] 2006, p. 15.) 8 The implementation of the Government Programme is monitored through the Government Strategy Document. It focuses on the themes cutting across the different administrative branches that are included in the Government Programme and on Government Policy Programmes. The Prime Minister’s Office publishes assessments of the Government Programme’s implementation in the middle and at the end of the Government term. Monitoring based on the Government Strategy Document has been conducted since 2003. The justifications of the Budget’s main titles of expenditure present the socio-political effectiveness goals that the ministries’ operations aim at. In addition to the Budget, the ministries’ operations are steered by their own strategies, operating and financial plans, annual performance plans, performance agreements concluded with public bodies or institutions, as well as by other official documents. The key documents related to the monitoring of the ministries’ operations are the sections that the ministries prepare for the Government’s Annual Report, their own annual reports, which also include the final accounts, as well as the sections the ministries prepare for the Report on the Final Central Government Accounts concerning the societal effectiveness of their own administrative branch in relation to the goals presented in the Budget. The Ministry of Finance steers and supervises the budgeting and financial planning of the other ministries. The Government’s joint controller function, which was set up on the basis of a recommendation of the Working Group on the Final Central Government Accounts (2003) and which operates in conjunction with the Ministry of Finance, assesses, supervises and develops national financial administration and its reporting systems. The controller function ensures that the reports on national financial administration and its performance contain appropriate and adequate information. It also coordinates and supervises the evaluation of administration. The National Audit Office is the supreme audit institution of state financial administration. It operates as an independent agency in conjunction with Parliament. The National Audit Office improves state financial administration by auditing the financial statements and performance. In a special report submitted to Parliament, the National Audit Office evaluates the application of the central government spending limits, as well as the appropriateness and adequacy of information submitted by the Government to Parliament. Publicising the objectives and results is an essential principle of the Government’s reform of performance guidance (Joustie 2009, p. 21). An Internet-based NETRA reporting system was set up to support this principle. Ministries are obliged to supply key information about their goals, finances and outputs to the system. Documents related to performance guidance are available to all citizens online. The NETRA database is maintained by the Treasury. 1.2 Indicators as tools for performance guidance and monitoring The Government’s reform of performance guidance and accountability systematised the use of indicators in setting goals and performance reporting (Indikaattorit [Indicators]... 2005, p. 25). As such, the basic notion is simple, as is stated in the said publication: indicators can be used to set specific goals in different policy areas and to monitor the achievement of goals. Specific and quantitative goals can even be set at the level of the Government Programme, for example, by agreeing that the goal is to raise employment by at least 100,000 jobs by the end of the electoral term. Usually, however, the goals of the Government Programme are qualitative and indicate the direction of development (ibid., pp. 23–24). According to the Ministry of Finance’s indicator working group, different levels of performance guidance based on societal effectiveness goals call for different types of indicators (Indikaattorit. [Indicators].. 2005, p. 4). The selection of indicators should reflect the hierarchies of the performance prism (ibid. p. 29). The Government Programme and the Government Strategy Document constitute the highest national level at which indicators 9 are utilised. Above this come international benchmarking indicators (such as those used by the EU, UN or OECD). The second highest national level is made up of the Budget and budget proposal, central government spending limits, operating and financial plans, as well as performance target documents. These are followed, at the third level, by output, outcome and quality indicators designed for the internal guidance of administrative branches and government agencies. The evaluation of societal effectiveness is challenging due to its expansive and multidimensional nature and the long timescales involved. According to the Ministry of Finance’s indicator working group, the lack of a theoretical framework causes problems in reporting (Indikaattorit.[Indicators].. 2005, p. 8). Broad-based societal effectiveness and operational performance get easily confused if the difference between targets and measures is not clearly defined. As such, the amount of publicly supported services does not provide information about societal effectiveness. Instead, the results should be examined from the perspective of broad-based socio-political targets. The causal relationship between administrative outputs and societal effectiveness is often difficult to prove. Firstly, this is a question of partial effectiveness in many administrative branches, and secondly, indirect impacts are difficult to perceive. The impact of socio-political decisions and measures needs to be distinguished from the impacts of all other factors affecting the phenomenon in question. Furthermore, attention must also be given to the temporal delay of impacts, which may span decades. Despite the challenges and problems involved in proving the societal effectiveness of different measures and sectors, the results of effectiveness assessments will most likely become more commonly used in the political administration system when planning the need to use society’s resources and their allocation. Indicative of this is, for example, the statement made by the Government’s policy work group on 24 February 2009, which urged the development of effectiveness assessments of policy measures. In October 2009, the Prime Minister’s Office also adopted Findikaattori, an internet portal with approximately 100 indicators on social progress, which more emphatically focuses attention on the top level of the socio-political performance prism. 10 2.1 Proof of evidence a challenge in cultural policy In Finland, cultural policy is supervised by the Ministry of Education and Culture. As the Ministry of Education and Culture, similar to other ministries, has increasingly begun to emphasise the strategic development of it administrative branch, management by knowledge has taken on a more important role. The working group on delegation and regionalisation (Tehtävien delegointi- ja alueellistamismahdollisuudet sekä ydintoiminnan kehittäminen opetus- ja kulttuuriministeriössä, [Options to delegate and regionalise duties and develop core operations in the Ministry of Education and Culture] (2003)) recommended that the Ministry of Education and Culture adopt the principle of strategic knowledge management. According to the proposal, the Ministry should develop duties supporting strategic decision-making in its administrative branch, as well as assume more responsibility for planning, organising, developing and utilising knowledge production in its sector. Strategic action and management by knowledge play a key role in the Ministry of Education and Culture’s development programme for 2007–2011. Similar to other administrative branches, cultural policy must report on its outputs and effectiveness within the framework of evidence- based policy-making and performance guidance. The culture sector has felt this to be problematic 2 Assessment of effectiveness in cultural policy for various reasons. The sector and its activities are of such a nature that they are difficult, and sometimes impossible, to characterise, especially with quantitative indicators. It is difficult, perhaps even impossible, to agree on unambiguous definitions of basic cultural concepts, such as culture, art and creativity. Cultural activities are dynamic and procedural. Many cultural impacts are relative, seeing as they involve human experiences and the assigning of meaning. It has been considered that this makes it more difficult to identify and compare impacts. Moreover, the causal relationships of cultural impacts are often difficult to detect. The problem is that the lack of data, analytical tools and resources often force assessments to focus on a small part of a bigger entity, which in itself is difficult to perceive. This blurs or distorts the picture of culture. It is also important to remember that the impacts in the culture sector may differ considerably and be difficult to anticipate, in addition to which they may be slow to emerge. (See Häyrynen 2004.) The vagueness and cumulative nature of goals have been mentioned as problems of cultural policy internationally. According to Bonet and Négrier (2003, p. 110), new cultural policy goals are typically set on top of previous ones. In their view, the cultural policies of western democracies have evolved cumulatively over the past century, 11 Cumulative evolution of cultural policies in western democracies Period Dominant rationale Intervention sectors Final goals Examples of indicators 19th century to 1945 Patriarchal, elitist and nationalist Archaeological patrimony, libraries and archives, fine arts Conservation, construction of the national identity, public instruction, art education of the elite Protected monuments, national distribution of school books, educational level, number of art schools 1945–1960 Artistic Fine arts and contemporary art Diffusion of high culture, creativity, quality and freedom of expression Number of quality institutions and festivals, seminal art performances 1960–1980 Socio-cultural Traditional culture, diversity and pluralism, mass culture, popular culture, means of communication Cultural democratisation, social integration and cohesion, cultural participation and democracy Social and territorial decentralisation of supply, inter-class participation, activities by non-professionals 1980–2000 Economic and cultural development Audiovisual sector, natural and intangible patrimony, new communication technologies, multimedia Domestic cultural production and diffusion, full employment, competitiveness and growth, multiculturalism, cultural cooperation Percentage of GDP, export capacity, supply of multicultural education, balanced international exchanges Source: Bonet and Négrier 2003, p. 110. as depicted in the table below. Their presentation also includes key indicators for different periods of cultural policy development. The details of this framework and its suitability for the Finnish setting can be debated, but the basic notion of the different development stages, or rationales, of cultural policy existing side by side is most likely true and also corresponds to Finnish descriptions of cultural policy development stages (see, e.g., Heiskanen 1994; Kangas 1999). The fact that some of the cumulative goals are at least implicitly conflicting and reflect different ideologies, which Kangas (1999, 176), among others, refers to, also explains why work involving indicators is difficult in this particular administrative branch. One of the recurring questions in the setting of goals in art and cultural policy is the choice between or balancing of a normative or relative notion of culture (see, e.g., Häyrynen 2006, pp. 105–106). 12 The obligation to verify the effectiveness of operations subjects administration to transparency, as is explained in the report prepared by the Ministry of Finance’s indicator working group (2005, p. 3). Apart from being a threat, this can be seen as an opportunity to increase visibility and to secure a foothold in the cultural policy sector. The idea has also emerged in international debate, which has encouraged parties to seize the opportunity to improve the status of art and culture (see, e.g., Madden 2005, p. 217). The challenge to demonstrate societal effectiveness may be even more interesting in a multi-faceted sector and administrative branch such as cultural policy. New perspectives on developing the sector and specifying the setting of goals may arise if comparable effectiveness assessment criteria, assessment frameworks and individual indicators are considered jointly with other administrative branches. Those working in the administrative branch of cultural policy feel that the sector is often relegated to a socio-political role that is smaller than its true significance. The experience has been that cultural topics and cultural policy are ignored when discussing socio-politically important matters (cf. the Council of Europe’s report In from the Margins 1996). The development of effectiveness assessments, with all its challenges, may be one way to ‘come in from the margins’. This means that the issue would no longer revolve only around the socio- political significance of cultural policy, but would emphasise the entire cultural dimension of social development, which has been considered important ever since UNESCO joined the group of cultural policy players in the late 1960s. So far, it has proved difficult to make the concept of broad-based culture more concrete in politics (see, e.g., Pirnes 2008). Efforts were made, for example, in connection with the UN World Decade of Cultural Development 1988–1997, when the relationship between culture and development was defined such that culture is considered a factor that inspires development instead of only being a consequence of it. The goal has been to give culture, as a dimension of development, a role alongside economic and social development. In Finland, the fundamental cultural nature of social development was expressed, for example, in the 1992 report on cultural policies (KUPOLI) and in the parliamentary cultural policy report based on it published the following year. KUPOLI (1992, 245) outlined a vision for the development of cultural policy and, consequently, social policy, which was based on a society of culture where the cultural values adopted by people permeate society, guiding decision-making and actions. The work of the arts and artists policy committee, completed in 2002, as well as the 2003 Government resolution on the arts and artists policy, based on the committee’s work, expound on the wide-ranging impacts of art on the development of individuals and communities. Culture and art are also described as being key factors in renewing and developing the foundation of social innovation. As an opportunity for self-development, art is considered to be a fundamental cultural right, as expressed in the Constitution of Finland (Valtioneuvoston. [Government...].. 2003, p. 6). Moreover, the resolution expresses the goal of giving culture more influence in social policy. The ‘in from the margins’ principle is explicitly mentioned in the discussion on how to improve the use of culture in regional development (ibid. p. 8). A stronger integration of the cultural dimension in socio-political decision-making came up in the Ethics of Cultural Policy project initiated by the Ministry of Education and Culture in 2005. The report by Hannele Koivunen and Leena Marsio (2006) discusses cultural rights from various perspectives. It also describes international initiatives to develop indicators on the ethics of cultural policy. The report covers cultural policy from perspectives such as diversity, the politics of identity, globalisation and sustainable development. The ethics of cultural policy were ultimately described using the concept of ‘fair culture’. Fair culture means the realisation of cultural rights and the inclusion of people in cultural signification, irrespective of their age, gender, minority group, language, ability, and ethnic, religious or cultural background. Two of the measures proposed in the report dealt with indicators. One of these recommended that a project 13 be set up in Finland from 2007 to 2010 to develop indicators on the ethical assessment of cultural policy. The other proposed that the development of Fair Culture indicators should constitute Finland’s initiative in the European Union’s Year of Cultural Dialogue, which was celebrated in 2008. However, these recommendations were not put into practice. In the future, the importance of culture as a force and dimension of social development will be a point of interest in cultural policy and a tool used to mainstream the cultural perspective. In the Swedish Government’s recent cultural policy report (Kulturutredningen 2009, p. 95), this issue has been highlighted under the term aspect policy. UNESCO and the Council of Europe will continue to advance the themes of culture and development, as well as culturally sustainable development. Both organisations are also working on indicators related to these themes, which will enhance international awareness of these issues through comparative analysis. New emphases – the cultural dimension of development, as well as the development of the culture industry and creative sectors – have typically found their way into Finnish cultural policy through discussions within UNESCO, the OECD, the Council of Europe, the EU and other international organisations. International recommendations and agreements are nationally binding. One of the latest examples is the Convention on the Protection and Promotion of the Diversity of Cultural Expressions, which entered into force worldwide in March 2007 (also known as the Cultural Diversity Treaty)1. 2.2 Assessment of cultural impacts and the effectiveness of cultural policy Cupore, the Finnish Foundation for Cultural Policy Research, has published two works related to the background, theories and methodology of cultural policy indicators: Kulttuuristen vaikutusten arviointi kulttuuripolitiikan toimenkuvana [Evaluation of Cultural Impacts as a Function of Cultural Policy, 2004] written by Simo Häyrynen, as well as Kulttuurin arviointi ja vaikutusten väylät [Evaluation of Culture and Channels of Impacts, 2005], edited by the aforementioned. Both publications were put together by Cupore on an initiative of the Ministry of Education and Culture, and they are considered to constitute the preliminary work for the recent indicator project. In his publication (2004), Häyrynen discusses the assessment of cultural impacts as an operating model comparable to the assessment of environmental impacts. The Act on Environmental Impact Assessment Procedure entered into force in Finland back in 1994. While cultural impacts are not synonymous with the effectiveness of cultural policy, they do have some mutual connections. Other policy fields may also have cultural impacts, just as the impacts of cultural policy may be other than purely cultural in nature. The assessment of cultural impacts can be linked to what was stated above on the realisation of cultural development in society. This means that the assessment of the effectiveness of cultural policy builds on a broad-based concept of culture and that such an assessment can in itself be understood as an activity in the field of cultural policy. Proposals concerning the evaluation of cultural impacts have been made in recent years, but a practical framework and measures have yet to be created. Cultural impacts have been included in the assessment of environmental impacts to some degree. According to the Finnish Act on Environmental Impact Assessment Procedure, environmental impacts of a cultural nature include, for example, the direct and indirect impacts affecting living conditions and comfort, community structure, buildings, landscape and cultural heritage. Correspondingly, proposals have been made both in Finland and abroad concerning the assessment of social impacts, health impacts and impacts on human beings. Cultural themes have mainly been associated with the latter. (See Häyrynen 2004, pp. 19–25; Kangas 2005, pp. 113–116.) 1 Finland was actively involved in the preparations of the treaty and was one of the first EU member states to ratify it in June 2006. The European Community ratified it in December of the same year during Finland’s EU Presidency. 14 2 http://www.minedu.fi/lapset_nuoret_perheet/Seuranta_ja_arviointi. Cultural impacts are sometimes evaluated in the Government’s cross-administrative policy programmes. Of the sitting Government’s policy programmes, the programme for the well-being of children, youth and families has adopted indicators such as the use of children’s culture and art services and the art hobbies of young people. They are related to the objective of preventing social exclusion2. Democracy indicators produced in connection with the Citizen Participation Policy Programme, one of the preceding Government’s programmes, include, under the theme of ‘informed citizenship’, matters such as municipal education and culture expenditure per capita and library lending figures per capita. In the array of sustainable development indicators, compiled by the Ministry of Environment and which encompass several administrative branches, culture services are mentioned in connection with the share of consumption expenditure of households accounted for by services, while libraries are mentioned in the examination of the distance of services from the home. Culture, defined in broad terms as the human development index (HDI), is one of the top key indicators of sustainable development (www. ymparisto.fi; more about HDI on page X of this report). Evaluations carried out in cross-sectoral activities can lead to important progress in the assessment of cultural impacts and in highlighting the cultural dimension. However, to understand cultural impacts as a whole, we need a more multi-faceted and systematic approach focused specifically on this theme. Under this basis, the next phase of cultural policy evolution could turn into a period of true cultural development (cf. Bonet and Négrier 2003). From the perspective of cultural policy, this would mean a shift from the assessment of individual aspects of cultural development to that of the big picture and, from the perspective of social policy, a shift from the analysis of individual dimensions of social development to an examination in which culture would be treated as a key dimension of development. The assessment of cultural impacts or the impacts of cultural policy remain vague if insufficient attention is given to defining the concept of culture. A specific, appropriate and transparent definition provides a framework within which assessments are carried out and for the themes on which they focus. In its broadest sense, culture can be understood to encompass all forms, processes and outcomes of human activities. The administrative branch of cultural policy obviously does not cover all of this. Moreover, a definition as broad as this would most likely create insuperable problems also in the assessment of cultural impacts. Under a slightly narrower interpretation, culture could be considered as being an essential element of the identity and lifestyles of people and communities or it could be defined as consisting of creative significations, actions and practices in social environments. None of these views offers a full match with the administrative branch of cultural policy, but they could function as viable starting points in the assessment of cultural impacts. In Finland, the field of cultural policy and administration has traditionally been linked to the arts and cultural tradition, both of which are still at its core. Today, the scope of cultural policy activities is wide, ranging from citizens’ opportunities to participate in culture and undertake independent activities to the economic significance of culture. As illustrated by the periods of cultural policy evolution defined by Bonet and Négrier (2003), the emphasis on citizens’ participation and independent approach dates back to the 1960s and 1970s. In the 1980s, economic impacts began to be increasingly used to explain cultural policy. The participation and well-being of citizens, as well as cultural diversity have taken centre stage in the 21st century. To an increasing degree, culture is now considered to be an industry and an inherent part of the national economy. Technological development and the adoption of new media have also modified the field of cultural administration. In practice, the administrative field of cultural policy consists of artistic and other creative activities and their impacts 15 on the experiences and lifestyles of individuals and groups, as well as on society and the national economy. What is more, cultural policy defined as actions affecting cultural development is not synonymous with cultural policy defined as an administrative branch. Cultural policy can be understood to encompass all of the players and actions that influence cultural activities and development (depending on the way in which the concept of culture is defined). This includes all parties with the power to define and execute cultural policy, as well as the means used to influence the functions and practices targeted by the objectives and measures. In central government, Parliament and the Government as a whole are considered to be cultural policy actors. They steer the implementation of cultural policy through legislation, the Government Programme, the Budget and other policies and decisions. Within Government, the responsibility for cultural policy implementation has been assigned to the Ministry of Education and Culture, which is divided into the Department for Education and Science Policy and the Department for Cultural, Sport and Youth Policy (with the third department being the Administration Department). National cultural policy is mainly the responsibility of the Department for Cultural, Sport and Youth Policy. For example, the Budget class dealing with art and culture (29.80) comes under the Department’s administration. Art education, as well as education and research in the fields of art and culture, which are connected to cultural policy, are handled, on the other hand, by the Department for Education and Science Policy. They are essential in creating a foundation for the professional development of the cultural sector and citizens’ cultural participation. Youth policy, which is the responsibility of the Department for Cultural, Sport and Youth Policy, also includes tasks related to culture. The Ministry of Education and Culture is not directly responsible for the performance guidance of all other cultural policy actors and neither does it have other financial and administrative connections to them3. In addition to the Ministry of Education and Culture and the government agencies under it, the key cultural policy actors include municipalities as well as actors in the third sector and individual citizens.Municipalities – often in cooperation with third sector actors – are largely in charge of the offering and infrastructure of regional and local art services, in addition to granting support for local cultural activities. The Ministry of Education and Culture steers municipalities through legally binding norms (Acts, decrees and decisions), resource control (statutory government contributions, discretionary transfers), information guidance (training, guidelines, manuals, letters) and the supervision of legality. The State and municipalities have joint administrative responsibility for the effectiveness of cultural policy in Finland. The impacts are often more visible at the local level. Third sector cultural policy actors include associations and foundations maintaining cultural activities and institutions, as well as foundations supporting culture and the arts. As foundations have boosted their role as cultural funders in recent years, their influence on cultural policy has also strengthened. The amount of support distributed by investment foundations largely reflects changes in the national economy and the development of stock markets. In the field of support for artists, for example, funding granted by foundations came close to that of the Arts Council of Finland towards the end of the first decade of the 21st century. (Oesch 2008.) The role of market actors can be roughly estimated on the basis of household cultural expenditure, which is quite notable (around four million euros in 2006, according to Statistics 3 The government agencies under the performance guidance of the Ministry of Education and Culture include the Arts Council of Finland, the National Art Councils and Regional Arts Councils, the Finnish National Gallery, the National Board of Antiquities, the Library for the Visually Impaired, the Governing Body of Suomenlinna, the National Audiovisual Archive, the Finnish Board of Film Classification and the Finnish Institute for Russian and East European Studies. The performance agreements made with govern- ment agencies define the agencies’ operational targets within the scope of the appropriations granted. 16 Finland), since most of it takes the form of market consumption. Mass media expenditure (such as newspaper and magazine subscription fees, televisions and set-top boxes, TV licence fees, pay-television fees) accounts for a notable share of this. Market-based press and digital media exert considerable influence on cultural policy. However, corporate art and culture sponsorship is not particularly extensive in Finland and seems to have further decreased of late (Oesch 2005). Ministries other than that of Education and Culture also carry out duties that affect the development of culture both in its broader and narrower meaning. The culture sector is influenced, for example, by the Ministry of Employment and the Economy, whose measures support the creative economy and related research and product development. The Ministry is also responsible for regional development, in which culture plays an increasingly important role. The Ministry of Transport and Communications is responsible for legislation governing television and radio broadcasting, as well as the communications market. It is also responsible for issues concerning the radio, television and other forms of communication, which many other countries assign to the Ministry of Culture and cultural policy. The Ministry of the Interior is in charge of matters related to immigration, a topic of growing significance in an increasingly multicultural society. The Democracy and Language Affairs Unit of the Ministry of Justice promotes civil society and especially the activities of NGOs, in addition to advancing public participation as well as research and reporting related to these topics. Cultural tasks and aspects can also be found in employment, social, health, economic, tax, foreign, trade and environmental policies. The areas of responsibility that are obviously shared with other administrative branches include issues related to the built and cultural environments, cultural exports and cultural tourism. Welfare policy is closely linked to cultural policy, while many cross-sectoral forms of ‘new policies’, such as innovation policy or sustainable development policy, can be analysed through their cultural dimensions or content. 2.3 Indicators in politics and cultural policy 2.3.1 Indicator classes and selection criteria Indicators are pointers or alarms that reveal a change in the state of affairs. They are statistical indicators of a kind that condense an increasing flow of information into a form that is easier for users to control and understand. Indicators describe the essential dimensions of phenomena and show the direction of their development. The objective is to measure progress towards the target state (or the approach of an undesirable state). A distinction is often made between a statistical figure that describes the situation in neutral terms and an indicator that measures the achievement of a goal. Individual statistical figures or compilations of them can, however, function as indicators if they are contextualised and interpreted from the perspective of goals. A statistical indicator is typically expressed as a ratio or percentage, and it often involves the use and comparison of a time series. The most ‘refined’ indicators, which often demand long-term development, are those that summarise data in a single figure (such as GDP or HDI, the Human Development Index). However, they risk being ambiguous and non-transparent. Barometers based on surveys as well as maps that enable geographical or other comparisons can also be used as indicators. There are many different classes of indicators, though no single, generally accepted categorisation exists. Distinctions can be made on the basis of the technology and methodology involved or based on the purpose of use. The difference between simple and composite indicators was explained above. Another frequently used distinction is that made between quantitative and qualitative indicators, although no general agreement has been reached on the meaning of this. The division into qualitative and quantitative indicators can be based on the calculation method (cf. qualitative and quantitative research methods), the scale of variables (categorical 17 and ordinal versus interval and ratio4) or the target of the measurement (quality or quantity). A qualitative indicator can be a verbal description, evaluation or classification, or it can be understood to mean a survey of human experiences and opinions. Sometimes a qualitative indicator refers to an indicator that evaluates quality and is expressed verbally or numerically. It is also possible to contrast quantitative indicators and qualitative evaluations, the latter of which can be carried out, for example, using an expert panel. Objective and quantitative aspects can be combined and contrasted with subjective and qualitative aspects. This is the case, for example, when satisfaction with services is considered to be a qualitative indicator (versus the number of service points, staff, opening hours or the average queuing time of customers). When considering the purpose of use and usability, it is possible to talk about background and guiding indicators or monitoring and performance indicators. Policy indicators are special types of indicators that are openly bound to specific political goals and strategies. They are tools used to set socio- political goals, plan measures, conduct monitoring and make decisions. The purpose of a policy indicator is to assess the change brought about by public intervention towards a defined goal. An indicator linked to a goal measures the achievement of the goal or the change in effectiveness resulting from a policy or programme. Indicators of this type can only be set if the objective of the policy or programme is known and if it can be clearly expressed. The Ministry of Finance’s indicator working group (2005, p. 4) lists relevance, validity and reliability as the key requirements for indicators used in performance guidance and accountability. These are general criteria related to statistical operations, research and assessment. By relevance the working group (2005, p. 38) means that an indicator — or a group of indicators or a composite indicator formed from individual indicators – truly reflects the core of the intended phenomenon. The validity of an indicator, on the other hand, means that the indicator measures exactly what it is intended to measure (which may also be an irrelevant feature of the phenomenon). A reliable indicator is statistically reliable in terms of the measurement (but may still be irrelevant or invalid in terms of content) (ibid.). According to the indicator working group, (2005, p. 38) indicators must be able to describe development but also look into the past in order for them to be suitable as tools for socio-political guidance. The indicator framework calls for theoretical clarity, while the concepts used and their mutual relationships must be precisely defined. When dealing with the performance prism, the goals concerning social development and the indicators describing them must be distinguishable from policy tools and the indicators describing them (ibid. p. 4). The relationships between functional near-term or interim goals and long-term social goals must be clear (ibid.). The indicator working group (2005, p. 38) sets the following requirements for a good indicator in the performance prism framework: • a small number but a wide scope • understandability • avoidance of overlaps • easy access • international comparability • a clear connection with policy goals. The Government financial controller (2006), on the other hand, sets the following requirements for a good (policy) indicator5: • It must be clearly linked to policy goals and tools. • It must describe the core of the intended phenomenon (relevance to guidance and management). 4 A variable is categorical or nominal , if the values it receives can only be categorised but not put into order of size. Examples of such variables are gender, fields of study, makes of car and countries of production. 5 The guidelines reflect the SMART criteria: specific = measures what it is supposed to (validity); measurable = the topic/theme can be measured; available = the information can be obtained at reasonable cost; relevant = relevant in terms of the measured target; time-bound. 18 • It ensures the valid measurement of the intended matter (validity). • It is understandable both in terms of content and the area of application. • It does not overlap with other indicators. • It is easily available. • It is comparable (preferably internationally). • It offers a sufficient view of the future (forecasting ability; development of the indicator as well as the sought or predicted development). 2.3.2 Special features in the selection of cultural policy indicators As specified in the assignment, this project focused on identifying key effectiveness indicators for the administrative branch by analysing the objectives of cultural policy and their mutual relationships. The focus of the indicator work was restricted to the reporting responsibilities and development projects of the Ministry of Education and Culture, which were defined during the reform of central government’s performance guidance and accountability, as well as to the Ministry’s own macro-level planning and monitoring needs. The criteria for effectiveness indicators, set by the Ministry of Finance and the Government financial controller, have been taken into account in this work. The project steering group wishes to emphasise that the indicators presented in this report reflect choices made from the perspective of current policy goals from a large group of existing or imaginable indicators describing social development and its sub- areas. At this stage, the availability of data and cost issues have played a critical role in their selection, in addition to their relevance from a cultural policy perspective. Given that indicators always simplify reality, it is obvious that the ability of this set of indicators to describe cultural policy as a whole must be evaluated critically and their further development must be continued. In the long run, an inadequate and outdated indicator system is problematic in view of the status of the administrative branch and socio- political guidance as a whole. Social activities cannot always be simplified in the same way to formulate commonly used and accepted types of indicator. Thus, they are not always directly comparable with one another. The critical question is whether the indicators selected in this project offer a sufficiently wide and versatile description of the operating environment and goals of cultural policy or whether they emphasise easily measurable but irrelevant issues. There is little material available that can be directly used in indicator work concerning the meanings, experiences and interpretations considered to be key in the field of art and culture. There is no unanimity on the concept of creativity even though the objectives of cultural policy include promoting creative work (artistic activities) and, more widely, all kinds of creativity in society. All this makes it difficult to measure the progress of creativity in any other way but indirectly, for example, based on the amount of appropriations granted. International discussions have also highlighted the risk that evidence-based decision-making might further emphasise instrumental goals in the field of cultural policy (see, e.g., Selwood 2006, p. 36). What the question ultimately boils down to is how big a role art and culture as such or different types of activities can have in relation to broad-based socio- political objectives. The measurement of ‘cultural value’ involves many problems from an instrumental perspective. Characteristics of public goods are often associated with culture. It is nearly impossible to prove how much added value culture or the cultural dimension alone produce in exchange processes that are determined on a financial basis, and it is even more difficult to do so when dealing with social processes. The broader the concept of culture, the more multi-faceted the problem becomes. The relationship between quantity and quality constitutes a particular challenge when evaluating the effectiveness of cultural policy. According to Häyrynen (2004, pp. 27–28), in the bulk of assessment research conducted to date, the effectiveness of culture and cultural policy has been described using statistical methods, meaning that culture has been treated as numerically measurable. Evaluations based on statistical data are often less 19 complicated to carry out than qualitative assessments and easier to generalise. It is also relatively easy to find comparison data for such evaluations. Moreover, people often trust statistics more than other materials in socio-political discussions and decision-making. According to Häyrynen, statistical classifications may, however, result in the loss of the special features of artistic work and activities. Statistics have a tendency to focus on formally organised activities and established classifications, even though the field of culture and the interactions within it change continuously and unpredictably. In Häyrynen’s opinion, this means that the assessment of the impacts of culture and cultural policy should not be based on statistical methods and materials alone. The indicator framework used for national performance guidance and accountability includes indicators that measure quality, since it is one of the key attributes emphasised in the reform. Quality may refer to, for example, the quality of public services from the customer’s point of view, in which case it can be measured using opinion surveys (customer satisfaction) (Indikaattorit.[Indicators].. 2005, p. 25). In the case of cultural policy, however, quality is a more multi-faceted challenge and is essentially related to core issues concerning the sector and the nature of its activities. It often seems difficult to measure quality with methods other than qualitative ones. Häyrynen (2006, p. 172) points out that statistics – and quantitative indicators in general – tend to marginalise the qualitative selection essentially related to cultural policy. The goals of cultural policy include promoting the arts and the operating conditions of both artists and other actors in the culture sector; advancing citizens’ participation, inclusion and community spirit; supporting cultural activities and the economy created around it; as well as protecting cultural heritage. It is not all that simple to measure the achievement of goals, since, as mentioned in chapter 2.1, assessments must take into consideration the diversity and long-term nature of cultural impacts. At their longest, the periods may span centuries or millennia and even at their shortest, years or decades. This, along with other issues related to the nature of culture, should also be taken into consideration when assessing the effectiveness of cultural policy. According to Häyrynen (2004, p. 30), it can be difficult to examine the diversity and subjectivity of impacts affecting human beings with quantitative indicators and statistics. The generalisation of assessment results also involves challenges caused by the diversity and versatility of cultural activities. However, diversity is one of the core objectives of cultural policy, while cultural experiences help to construct cultural identities, individual empowerment and life management, as well as social relationships. It is important to identify differences between cultural groups because cultural activities, and especially participation in institutional cultural offerings, are manifested in different ways in different groups. Cultural tastes and divisions into fields of art are other issues related to the challenges of assessing the effectiveness of cultural policy that call for qualitative evaluations. The question of the quality of art introduces a special challenge to the effectiveness assessment of cultural policy. In line with the nature of its role, the Ministry of Education and Culture does not put a value on the content of art and culture. Instead, in accordance with long cultural policy traditions, the task has been assigned to the field itself to be conducted on the basis of peer assessment. If required, the functioning of this arrangement can be assessed within the framework of the administrative responsibility for cultural policy held by the Ministry of Education and Culture. When examining quality in effectiveness assessments, the Ministry of Finance’s indicator working group (2005, p. 32) found that qualitative considerations can also be quantified by converting them into figures corresponding to school grades. Even when converted into figures, they still remain estimates and partly subjective. This interpretation may be theoretically valid, but in practice, problems may arise if quantified qualitative assessments take on a life of their own and the thinking behind them is forgotten or no longer taken into consideration when applying indicator data in different contexts. 20 Qualitative assessment methods Quantitative assessment methods Describe issues that are difficult to split into measurable elements (happiness, good life, enjoyment) Verbal description of qualitative matters (such as the impacts related to happiness, enjoyment or community spirit) Qualitative assessments provide answers to questions such as ‘what’, ‘where’ and ‘how’ Describe changes that can be expressed as measurable attributes (euro, dB or days in care) Quantitative assessments provide answers to questions such as ‘how much’ and ‘when’ Examples of methods: Analysis of interactive material Interview Negotiations/discussion Delphi Examples of methods: Cost-benefit analysis Measurable indicators Diffusion models Questionnaire Source: Kauppinen and Tähtinen 2003, p. 18. When discussing the interpretation and usability of indicators, it may be useful to take into consideration the analysis of Häyrynen (2004, pp. 44–51) concerning the identification of effectiveness, which was presented in the context of cultural policy but is also applicable in broader terms. According to Häyrynen, certain features of phenomena are experienced as being positive or negative in this context, while others are totally ignored in political and administrative observations. For example, the negative environmental impacts of cultural events or cultural tourism are not brought up in the same way as those of other types of event production and tourism. These observations are of significance when evaluating the interest of social players to demonstrate the positive effectiveness of their own activities in situations where effectiveness assessments are used as criteria for distributing the resources of society. In this project, the focus of attention was basically restricted to indicators immediately available in central government monitoring systems, which means that the selected cultural policy effectiveness indicators do not cover the entire field and sector of cultural policy – let alone evaluate cultural impacts in the sense described by Häyrynen (2004). Using his classification, the definition chosen for the project could be characterised as ‘assessment of the immediate functionality of the administrative sector’ (p. 9), ‘internal assessment of cultural administration’ (p. 18) or ‘administration-oriented assessment’ (p. 19). The question of the diverse and often indirect social effectiveness of culture led to such extensive discussions that it was impossible to cover all of the issues involved in depth within the limits of the assignment, even though they were discussed at the steering group’s meetings and in workshops. One of the essential guidelines of the cultural policy indicator project was the requirement that indicators should be transparent. Indicators must be methodologically reliable and objective, in addition to which their commitment to specific policy goals must be clearly indicated. The purpose of indicators is to make political decision-making and the results of activities carried out by the administration implementing the decisions transparent, which is one of the key principles of the reform of the performance guidance system. This requires that the connection between cultural policy goals and each individual indicator, as well as the whole group of indicators, is expressed as clearly as possible. According to this criterion, indicators must be sufficiently simple and unanimous, as well as easy to interpret and communicate to extensive social circles. The formulation and selection of cultural policy indicators also emphasise the resources perspective. To ensure continuity, information must be available 21 and updatable at moderate cost. Many of the selected indicators are of the kind that are already produced, for example, in connection with the creation of cultural statistics by Statistics Finland or for which the required data material could be obtained with minor additional work. The criteria for economic efficiency and availability led to the cultural policy indicators presented in the project being mainly quantitative. The ultimate goal, however, is to also develop qualitative indicators and assessments to describe cultural activities and the culture sector. Indicators that would be useful for the planning and monitoring of cultural policy, but with respect to which not enough information is yet available and the obtainment of which would require considerable additional resources or qualitative research and investigation, have been formulated as proposals for development. The assessment of cultural impacts and cross- sectoral policies, as well as cross-sectoral activities such as the policy programmes included in the programmes of the past Government term, can be seen to present opportunities for cultural policy assessment, and for the development of cultural policy as such. Based on the title of his report discussed above (Kulttuuriset vaikutukset kulttuuripolitiikan toimenkuvana [Cultural impacts as the Function of Cultural Policy], Häyrynen (2004) implies that cross-sectoral assessments should be adopted as part of the range of cultural policy tasks. He considers cultural impacts to be crucial factors in the cultural policy operating environment (ibid. p. 54). In Häyrynen’s opinion, freedom from restrictions created by the division of tasks between administrative branches – what could be called ‘non-system-bound assessment of cultural impacts’ – could broaden administrative officials’ understanding of culture as a whole and their own role more broadly in social policy (ibid. p. 87). In this project, this approach is reflected in some of the development proposals presented in the report. They can be understood as tools for the mainstreaming of cultural policy. 2.4 Development projects focusing on cultural policy effectiveness indicators outside Finland and in international organisations 2.4.1 Cultural indicator boom and levels of examination To provide a foundation for the indicator project, similar projects evaluating the effectiveness of cultural policy elsewhere in the world were sought using literature and Internet sources. IFACCA, the International Federation of Arts Councils and Culture Agencies, which has surveyed indicator work carried out around the world and within international organisations in recent years, was also contacted. In 2005, IFACCA published its widely cited basic survey Statistical Indicators for Arts Policy. Christopher Madden (2004, 2005a, 2005b), who has worked as research analyst at IFACCA, has published several articles about the international comparability of cultural statistics and indicators. Indicators describing culture and cultural policy are currently being developed in many other countries around the world as well as within international organisations. What we are seeing is a worldwide indicator boom affecting all socio- political sectors (see Sauli and Simpura 2004). In terms of this wide-ranging phenomenon, this report focuses on the part that deals with the measurement of cultural policy impacts and effectiveness. In many countries, effectiveness indicators have been created in response to the verification obligation of public administration resulting from evidence-based decision-making. The shift from indicators describing input or resources to indicators portraying output and impacts took place in public administration back in the 1990s. It was influenced by ‘new public management’ as well as new policies in strategic management and performance guidance. Furthermore, it was partly strengthened by the recession of the early 1990s, which made the requirement for increasing productivity imperative and called for more detailed verification of achievements overall. 22 Indicator type Focus Culture examples Selected references Macro Generic Cultural indicators for development Baltà (2004) Mercer (2002) Indicators for evaluating national Nylöf (1997) cultural policy Matarasso (1997) Meso Indicators for evaluating art policies van der Ploeg (2004) Joy et al (2004) Performance indicators for South West Arts Marketing cultural institutions (2000) Pignataro (2003) Micro Specific Indicators for evaluating community arts programs Keating (2002) Source: IFACCA/Statistical Indicators for Arts Policy 2005, p. 23. IFACCA’s work on indicators deals expressly with indicators for art and cultural policy as opposed to indicators describing the culture sector or cultural life in broader terms. The Statistical Indicators for Arts Policy publication suggests that indicator work be initiated by considering the level at which impacts will be measured. Indicators are divided into those operating at the macro, meso and micro levels. At the macro level, the focus is on the wider impacts of cultural policy measures and, for example, the links between culture and development. At the micro level, assessment focuses on individual art institutions or programmes (ibid. p. 23.) The present indicator project deals with the macro and meso levels of this table, examples of which include articles by Jordi Baltà (2004), Colin Mercer (2002), Göran Nylöf (1997), François Matarasso (1997) and Rick van der Ploeg (2004). The articles of Baltà and Mercer are clearly at a more generic level than the approach adopted in the project of the Ministry of Education and Culture. Baltà discusses the development in Africa of cultural indicators on human development. The activities concerned relate to operations carried out within UNESCO. The themes covered by the indicators include cultural rights, equality, creativity and cultural entrepreneurship. In addition to statistics, the activities concerned the creation of databases and the dissemination of best practices. Mercer’s article is also linked to the relationships between culture and development and proposes indicators that can be used to measure links between cultural policy and human development. Mercer divides indicators into four themes: 1) cultural vitality, diversity and sociability, 2) cultural access, participation and consumption, 3) culture, lifestyle and identity and conviviality, 4) culture, ethics, governance and conduct. Mercer’s indicators are included in his report Towards cultural citizenship: Tools for cultural policy and development, prepared for the Bank of Sweden Tercentenary Foundation. The report is related to the UNESCO report Our Creative Diversity, published in 1995 in conjunction with the World Commission on Culture and Development. Mercer’s starting points are the concepts of cultural citizenship and cultural basic rights, as well as the principle of sustainable development. The indicators presented in the report are verbal descriptions of themes that should be discussed when assessing the influence of cultural policy in the advancement of human development. In his article, Göran Nylöf (1997) examines the methods used in country evaluations carried out since 1985 under the Council of Europe. Nylöf was personally in charge of the methodologies used 23 in Sweden’s evaluation (1992). Robert Wangermée (1993), who developed country evaluation methods as a consultant in the early 1990s, pointed out the need to create a reliable knowledge base for monitoring and evaluating cultural policy, placing special emphasis on the role of indicators. Wangermée highlighted the need to consider cultural policy goals and the related value sets when assessing the effectiveness of cultural policy methods, since the field involves more than a technical input-output analysis alone. (Mitchell 2002, pp. 6–7.) Finland’s country evaluation, for which a great number of statistical and indicator data was produced, was completed in 1995. By 1999, the evaluation had been carried out in 15 countries. Since then, the project has expanded to countries outside Europe, such as Vietnam. The country evaluations carried out within the Council of Europe have considerably advanced international development work on indicators evaluating the impacts and effectiveness of cultural policy. In relation to the project, Augustin Girard (1992) drew up an indicator report that made use of, above all, data on evaluations conducted in Sweden and France at the time. Girard presented the statistical data and their interpretation on the same page, the goal being for readers to first study the tables and then acquaint themselves with the related comments. Interpretation relied heavily on contextualisation, which required wide-ranging knowledge of the features of each country’s cultural, social and economic life. Above all, the statistical and indicator data had to be considered in relationship to the goals of cultural policy. The purpose was to measure the internal effects of cultural policy measures (input–output) as well as their ultimate effectiveness with respect to citizens’ cultural participation. (Girard 1992, p. 6.) In the IFACCA table, meso-level indicators are represented by the articles written by François Matarasso and Rick van der Ploeg. Matarasso has become famous especially for his articles dealing with the social impacts of culture, one of which is Use or ornament? (1997), which is mentioned in the table. Matarasso has been especially involved in developing indicators assessing cultural programmes at the level of local communities (e.g., 1996 and 2001). The publication of van der Ploeg (2000/2004) titled Cultuur als confrontatie is a document that has reformed the main principles of Dutch cultural policy. The author wrote it in 2000 when acting as State Secretary. The document defines cultural diversity, reaching an increasingly wide and diverse public, as well as cultural entrepreneurship as the priorities of cultural policy. The author used a great deal of statistical data to support his arguments. 2.4.2 International organisations and communities The compilation of cultural statistics started internationally in the 1960s and 1970s in the Council of Europe and UNESCO, which was expressly set up to serve the cultural policy sector that began to emerge at the time (see, e.g., Karttunen 2004). UNESCO’s cultural statistics framework, which has been subject to wide-ranging influences, was established in 1986 as the result of international cooperation. The development of indicators was also carried out in connection with the framework. Leif Gouiedo, from Statistics Sweden, for example, drew up several indicator reports (1985; 1993). Indicators were specifically seen as tools for cultural policy and planning, and they were to serve cultural policy decision-making and implementation throughout the process. Indicators were primarily expressed as different types of ratios. Gouiedo (1985, p. 28) divided cultural policy planning into the following phases: 1) survey of the situation, 2) decision- making, 3) implementation and 4) the evaluation of impacts (ibid. p. 28). The corresponding indicators were 1) to provide assistance in the preparation of decisions and 2) in the actual decision-making, 3) to monitor activities and 4) to measure the results. In practice, much of the statistics and indicator work carried out within UNESCO in the 1980s and 1990s focused on surveying situations, which meant conducting questionnaire surveys to identify citizens’ cultural needs and cultural obstacles. Indicators related to decision-making did not get much attention, and in the case of implementation, 24 the focus was primarily on public cultural budgets, as well as institutional networks and trends in the amount of services provided by them. As for output, most of the data collected dealt with the preservation of cultural heritage and the cultural participation of citizens. Indicators provided background information on and descriptions of the sector rather than offered strict evaluations of the outputs in relation to the goals. From the perspective of effectiveness assessment, what is noteworthy in Gouiedo’s model is that the measurement of results was restricted to the cultural sector (similar to Girard above). Outputs of cultural policy were considered to include, for example, growth in the cultural participation of citizens, but not the impact of cultural participation on, say, social cohesion. Structural differences between countries can partly explain the emphases of the international cultural statistics presented in connection with Gouiedo’s indicator report (1985). It would have been important to contextualise the data in different ways to enable comparisons between countries at least to some extent. It was for this purpose that UNESCO developed the concept of cultural policy databases. In Europe, the concept has been put into practice through the Compendium project, initiated by the Council of Europe in 1998. In addition to verbal descriptions of the cultural policy profiles of different countries, the database now contains a great deal of quantitative information about statistics and indicators. UNESCO’s cultural statistics framework (1986) has recently been modified to account for the changes brought about by globalisation and the development of the Internet. On the one hand, the new framework aims to describe the cultural economy to a greater extent than before and, on the other hand, it also tries to answer the needs of emerging countries, for example, by including intangible cultural heritage in the framework. The framework template was produced on the basis of a consultancy agreement in Great Britain, after which member states and professionals were consulted in 2008 and 2009. Final approval of the reform was scheduled to take place in 2009. The effectiveness assessment of public cultural policies has not played a key role in the reforms, nor has the development of indicators been discussed yet. To ensure the feasibility of international comparisons, information provision will rely on materials that use internationally harmonised classifications (sectors, professions, products etc.). Work is currently being undertaken within UNESCO to develop cultural diversity indicators and indicators for monitoring the Convention on the Promotion and Protection of the Diversity of Cultural Expressions, which came into force in 20076. In its report, Our Creative Diversity, published in 1995, the World Commission on Culture and Development proposed the drawing up of a world culture report. UNESCO published the first culture report in 1998 and another one in 2000. The extensive reports contain articles on the relationships between culture and development, written by several experts from various perspectives, as well as a separate section on statistics and indicators. The thematic topics of the first report were culture and the economy; cultural rights and ethics; creativity; markets and cultural policy; as well as public opinion and global ethics. The second report focused on cultural diversity, conflicts and pluralism. In these reports, culture is defined widely to encompass the lifestyles of individuals and communities. The preparatory document for developing cultural indicators for the world culture report was published by UNRISD (the United Nations Research Institute for Social Development) in 1997. In addition, an international seminar on indicators was arranged in connection with the Culture Counts conference, organised jointly by the World Bank, UNESCO and the Government of Italy in Florence in 1999. The person in charge of the indicators for the published reports was Leo Goldstone (World Statistics Ltd). The culture report and its statistical appendices were intended to be published every two years. 6 Final Report: Expert Group Meeting (EGM) on the Statistical measurement of the diversity of cultural expressions (2007). http://www.uis.unesco.org/template/pdf/cscl/cultdiv/mtgreport.pdf. 25 In the early 21st century, UNESCO concluded that the number of world reports on different sectors produced since the early 1990s was enormous in view of the organisation’s capacity. It decided instead to change over to a single annual world report with changing themes (UNESCO World Report), the goal being to gain greater visibility compared with previous reports. The UNESCO Institute for Statistics (UIS), founded in 1999, is today responsible for the statistical sections of the reports. The UNESCO World Report on Cultural Diversity has been in the making since 2006, and should be published in the near future. Work on human development indicators and the Human Development Index (HDI) has been conducted within the United Nations Development Programme (UNDP). Data have been published in Human Development Reports (HDR), which have been drawn up since 1990. The special theme of the 2004 report was ‘Cultural Liberty in Today’s Diverse World’. The term referred to cultural rights, people’s ability to live in accordance with their own identity and choices. The antonym of cultural liberty is political, economic and other types of exclusion related to one’s lifestyle, ethnic background, language or religion. The report contains both discussions about the assessment of the field and a great deal of figures and tables. It suggests that people and their rights should once again be put at the core of cultural policy. Indicators on macroeconomic impacts, international trade, investments, the production value chain , information technology and intangible rights, companies and tourism have been produced in connection with the Creative Economy & Industries programme of UNCTAD (the United Nations Conference on Trade and Development). The goal of the 2008 Creative Economy report is to offer evidence-based analyses of the economic significance of the creative industries. According to the report, the topic could be examined from the perspective of employment, the use of time, trade, added value and copyright. The development of indicators in this special field is still incomplete, however, and some of the information is not available even in developed countries. The Creative Economy report suggests that culture satellite accounts based on national accounts could be used to present information on the cultural economy for use in public and private decision- making. National accounts have been developed internationally under the lead of the UN and are used in most countries around the world. The report describes cultural satellite projects that the Convenio Andrés Bello organisation has initiated in Latin America. The worldwide diffusion of the method is expected to take several years, and the resources are not expected to suffice everywhere. Consequently, the report ends up measuring the economic significance of the creative industries on the basis of existing world trade statistics. The report’s extensive statistical appendix reviews the import and export of commodities and services provided by the creative industries in different countries around the world. One of the suggestions is that countries with sufficient data and resources should use both satellite accounts and trade statistics, which are considered to be complementary. International organisations have begun to pay more attention to national economy indicators and their inability to properly describe development and well-being. An example of this is the esteemed Commission on the Measurement of Economic Performance and Social Progress (also known as the Stiglitz Commission), which was established in 2008 at the initiative of France to consider the measurement of links between economic development, the quality of life and sustainable development. In addition to France, the Commission includes the OECD, UNDP, the World Bank and a great number of internationally well-known representatives of universities. The Commission published its report in September 2009. As such, culture has not been greatly emphasised in the report’s explanations. Nevertheless, the report draws attention to the amount of leisure time as an important indicator of well-being. Likewise, it emphasises the importance of education, talent and opportunities, as well as day-to-day activities, freedom of speech, social participation and social contacts for well-being. 26 The OECD, for its part, has for several years spoken in favour of expanding the description of social development. It also hosts a broad-based project known as Measuring The Progress of Societies. At the beginning of September 2009, the OECD published a draft of a framework for measuring the progress of societies. The proposed taxonomy includes sections such as knowledge and understanding, work and leisure, freedom and self-determination, cultural heritage, as well as arts and leisure. The OECD Statistics Directorate compiles economic statistics that are used as the basis for the work of the OECD, develops international statistical standards and cooperates with other international statistics agencies in statistical activities. The OECD produces some indicators related to leisure time. The Project on the International Measurement of Culture, carried out during the first decade of the 21st century, examined, among other things, ways to measure the economic and social importance of culture (see Gordon and Beilby-Orrin 2006). Unfortunately, due to insufficient funding, only the first phase of the project was carried out. The plans included assessing the economic importance of culture using satellite accounting and developing statistics in the fields of cultural participation and leisure time. The EU has also worked actively to improve the description of societal progress. In August 2009, it published a policy paper entitled GDP and Beyond: Measuring Progress in a Changing World. The goal is to complement GDP with environmental and social indicators; and the aim is to publish an environmental index annually. Components of well-being mentioned in the report include income, public services, health, leisure, wealth, mobility and clean air. The objective is to draw attention to inequality in these fields. Eurostat, the EU’s statistical office operating under the European Commission, produces and publishes statistics for the European Union and its member states. The indicators of the European Statistical System (ESS) can be divided into short-term Euro indicators, long-term indicators, structural indicators and sustainable development indicators. Culture does not play a major role among them. Eurostat released a cultural statistics publication in 2007. In addition, Eurostat and the European Commission’s Directorate-General for Education and Culture have commissioned a couple of Eurobarometer studies to provide information about cultural participation and the appreciation of culture (e.g., European Cultural Values 2007). A small module on cultural participation was also included in the 2006 EU-SILC (EU Statistics on Income and Living Conditions) and the EU Adult Education Survey. No actual work on cultural policy indicators has been performed yet within the EU, though it is true that, for example, cultural employment and cultural participation figures, expressed as percentages, can be used as indicators, especially if they are interpreted more closely in relation to their context. The European Commission has shown particular interest in the economic importance of culture and the creative industries within the Union. In 2006, it commissioned a study on the economy of culture in Europe. The study provided information about the sector’s revenues and share of GDP, which are considered to be key indicators in the field. Their links to cultural policy measures and their effectiveness are not analysed in greater detail in the report. A report on the impact of culture on creativity, commissioned by the European Commission, was published in summer 2009. The report’s aim was to consider, both theoretically and politically, the way in which culture-based creativity influences society’s economic – and to some extent its social – progress. It contains hardly any statistics or calculations. ERICarts (European Institute for Comparative Cultural Research) and the Council of Europe, which cooperate to produce the cultural policy Compendium mentioned above, have also created a framework for evaluating cultural diversity, social cohesion and intercultural dialogue as a part of the Compendium (Foote 2005). It could mainly be characterised as a descriptive list of questions, which is nevertheless called an indicator. The list functions as a normative tool for peer assessment. 27 2.4.3 Examples of national indicator projects The UK’s Department of Culture, Media and Sport ( DCMS) has created a special toolbox for collecting and presenting data required for evidence-based decision-making. It mainly consists of cultural statistics, which provide data on the situation and development trends that can be used when planning future policies. The Arts Council of England, which operates under the performance guidance of the DCMS, conducts annual surveys of organisations receiving regular funding. In 2005, the DCMS set up an internationally influential programme on the creative economy. It was based on sectoral mapping, which has also been adopted in many other countries, such as Singapore. Projects evaluating the compilation of statistics on the creative industries and the economic impacts of the creative sectors have been launched both nationally and regionally as a part of the creative economy programme. The debate about the ‘value of culture’ that has gone on in the UK in recent years is also interesting from the perspective of the cultural policy indicator project (see, e.g., Bunting 2007; Holden 2004). In short, the debate has focused on the need for public intervention and on social benefits in the field of culture. The main issue has been whether public support for culture can be justified in its own right or whether it always requires instrumental justification (see Selwood 2005, p. 116). Discussion topics have included the priorities in cultural policy, the attention given to the views of different interest groups and participants, as well as the quality of cultural services and products. Accountability and the demand for transparency have also been raised. It has been proposed that the value of culture be measured using surveys, barometers and indicators. According to Eleanora Belfiore (2004), the demands of New Public Management have strengthened the ‘instrumental turn’ in cultural policy in the UK. In addition to Australia and France, Canada is a world leader in the compilation of cultural statistics. The Observatoire de la culture et des communications du Québec (OCCQ) has developed an indicator system based on the work of IFACCA. The system, introduced in the publication Counting Culture in Quebec: A System of Indicators for Culture and Communications (2007), includes 14 key indicators, measuring either economic or societal impacts. The Quebec indicators are primarily related to cultural policy, but they are not designed only for use by public administration and cultural policy actors, but rather to serve all players in the culture sector. In other words, the indicators are not managed solely on the basis of official cultural policy goals. Furthermore, the publication explicitly denies any effort to measure the effectiveness of specific cultural policy activities (ibid. p. 31). A knowledge base project dealing with cultural policy has been carried out in Australia in the past few years. It also has points in common with the development of cultural policy indicators in Finland. The Australian indicator work has also been influenced by the requirements of the country’s Department of Finance. The project has been jointly carried out by the Bureau of Statistics and the Cultural Ministers Council and is closely linked to the design and monitoring of cultural policy. However, the goal is also to serve wider circles of users. In 2008, the Australian project resulted in a framework for research and the compilation of statistics (Arts and Cultural Heritage in Australia 2008). The ‘value of culture’ has also been a central theme in the debate in Australia, which is presented at the beginning of the publication (ibid. pp. 3–11). The ‘societal effectiveness’ and ‘public value’ of cultural policy appear to be closely related based on the introduction. 28 14 most important cultural indicators in Quebec Economic Indicator Objective 1. Ratio of the GDP of the culture and communications sector to total GDP Measures culture’s share of the economy. Cultural GDP measures the value of economic resources devoted to culture. Indicates cultural vitality. 2. Cultural products export index Measures the change in the value of cultural product exports. Indicates cultural vitality. 3. Cultural products price index Measures the change in the cost of cultural products. Indicates economic accessibility to culture. 4. Cultural industries concentration index (production and distribution combined) Measures the degree of concentration of the industry to help understand challenges in accessing the market. Indicates economic accessibility. 5. Total cultural products sales index Measures change in the value of cultural product sales. Indicates evolution in cultural consumption. 6. Share of the total cultural products market Evaluates the situation of Quebec cultural products in the total of cultural products sold. Indicates cultural vitality and, in the Quebec case, cultural diversity. 7. Make-up of total sales by producing country index Measures change in the extent of cultural diversity according to country of origin. Indicates the change in openness of Quebecers to other cultures. Social Indicator Objective 8. Ratio of the number of cultural workers to the total labour force Measures the evolution of the strength of the culture labour force. Indicates cultural vitality. 9. Number of distribution establishments per 1,000 residents Measures the accessibility of culture. 10. Ratio of household spending on culture to spending on leisure Measures change in the share of household spending on culture as a part of spending on leisure. Indicates cultural vitality. 11. Composite index of attendance at cultural establishments Measures the patrons of cultural establishments to indicate the growth in participation in culture. 12. Composite index of involvement with cultural activities Measures the prevalence of different cultural behaviour habits. Indicates cultural vitality. 13. Ratio of time spent on cultural activities to total leisure time Measures changes in the social time devoted to culture. Indicates cultural vitality. 14. Ratio of new works to the total supply Measures developments in new original content in culture supply. Indicates cultural vitality. Source: Allaire 2007, p. 13. 29 Source: Arts and C