Multiple Criteria Evaluation of Influence of Components of Entrepreneurship Education Programmes on Formation of Competencies

The article substantiates the concept of entrepreneurship education from the holistic perspective, which is interpreted in the context of education in its broad meaning and is related to knowledge, abilities of economics and business, value orientations, personal traits and their comprehensive development. The significance of content areas of the General Education Curriculum to development of entrepreneurship skills in schools of general education in Lithuania was analysed. Multiple criteria evaluation, based on expert evaluations, was applied for evaluation of significance of areas of the programme content mathematically processing and systemising the data. The significance of areas of programme content evaluated by experts was established using methods of indirect evaluation (ranking) and direct evaluation (percentage expression). The research showed that the data of expert evaluations, conducted using the methods of indirect and direct evaluation, coincide and, therefore, the data can be regarded as reliable. The conducted mathematical calculations show that multiple criteria methods can be successfully applied evaluating any other programme of similar kind.


Introduction
Relevance.Entrepreneurship is a priority trend in educational activity and is determined as one of the most important general competencies.The investments allocated to their development can bring one of the biggest returns (Entrepreneurship in the EU and Beyond: Report, 2012).
Entrepreneurship education is a relevant scientific problem, which evokes numerous discussions.The concept of entrepreneurship based on holistic approach has to be interpreted in a broad way and related to knowledge, abilities of economics and business, value orientations, personal traits and their importance in all the spheres of life as well as to their comprehensive development (Ruškytė, 2016).
The General Education Curricular are in a consistent transition towards new policy on curricular development, i.e. from knowledge conveyance to building up of competencies (knowledge and understanding, abilities and value orientations) focusing on increase of flexibility and versatility of educational programmes and school learners' choice, application of innovative education methods and aids considering school learners' interests, age, level of understanding and experience (Pradinio ir pagrindinio ugdymo bendrosios programos, 2008; Vidurinio ugdymo bendrosios programos, 2011; Pradinio, pagrindinio ir vidurinio ugdymo programų aprašas, 2015; Ruškytė & Navickas, 2015).The innovative content of entrepreneurship education is renewed taking into account changing educational needs and expectations of school learners as well as increasing variety of educational opportunities, strategic trends of economic growth, international educational tendencies, scientific and technological innovations (Novatoriška verslumo ugdymo samprata, 2016).
Problem.The generally acknowledged model for entrepreneurship education and its practical implementation has not been available up to now; therefore it is not clear how entrepreneurship can be developed best.The conducted research studies most frequently focus on the problems of entrepreneurship promotion rather than on entrepreneurship education.
As it is pointed out in the report of the research conducted in European schools in 2016, currently the absence of comprehensive learning outcomes related to entrepreneurship education is one of the main obstacles for formation of efficient and high quality entrepreneurship education (Entrepreneurship Education at School in Europe, 2016, p. 11).
The aim is to establish the significance of content areas of the programme of entrepreneurship education to school learners' entrepreneurship education.

The context of entrepreneurship education
In scholarly literature, the concept of entrepreneurship is discussed in different context and interpreted not uniformly.According to the researchers, the major confusions and uncertainties occur regarding the use of the concepts entrepreneurship, entrepreneurship skills, and entrepreneurship competency.Most frequently these concepts are used as synonyms but they are treated differently.This is most frequently related to the context of their use (Ruškytė, 2016, p. 67).Recently numerous discussions have been generated linked with the interpretations of the concept of entrepreneurship in its narrow and broad meanings.
The concept entrepreneurship is most frequently used in its narrow meaning in the contexts of economic nature and is determined as ability to take on risky actions to create economic added value and to gain profit, i.e., as an ability characteristic of a person, who establishes and develops own business.The concept entrepreneurship is often associated with a person, an owner or manager of a business enterprise, who seeks profit by taking initiative and putting his/her assets at risk.I. Komarkova, J. Conrads, and A. Collado (2015a, p. 4) state that in its narrow meaning entrepreneurship is linked with enterprises' activities and their functions.Entrepreneurship is a person's disposition and ability to undertake economic activities joining capital, work and other economic resources seeking profit and assuming all the related risk (Vainienė, 2008, pp. 282-283).Entrepreneurship is ability to apply possessed knowledge in personal life, to make risky decisions to set up and develop own business (Župerka, 2010, p. 137).
In a broad meaning the developed personal traits are associated with entrepreneurship, which allow to create not only an economic value but also social and all the other values (Zaleskienė & Žadeikaitė, 2008;Palčiauskienė & Virketytė, 2009;Čiburienė & Guščinskienė, 2009;Paulionienė, Rakauskienė & Durtinevičiūtė, 2011).Entrepreneurship embraces personal qualities such as initiative, creativity, self-confidence, a fast reaction to certain changes, a critical attitude, logical thinking, intuition and pursuance of career.Entrepreneurship embraces a wish to change something himself or herself and ability to accept and support innovations, to assume responsibility for own actions, to finish what has been started, to be aware of direction, to set up goals, to expect to achieve them and make attempts to do so.Entrepreneurship is characteristic of individuals, who are pro-active, show initiative, are curious, possess firm intrinsic motivation, inclination to innovations and are ready to take risks.
Entrepreneurship is determined as a way of thinking and personal, social and managerial competencies that make it possible for an individual to apply the possessed knowledge in daily life, i.e., as specific abilities that enable not only to organise own business but also to assume risk for the decisions made (Ekonominio raštingumo ir verslumo ugdymo strategija, 2004).Thus, in its broad sense entrepreneurship is related to the thinking and behaviour of the learner (Komarkova et al., 2015a, p. 4).
Entrepreneurship is ascribed to eight general competencies that are necessary for personal satisfaction and development, active citizenship, social integration and employment.It is defined as an ability to turn ideas into actions, i.e. as initiative, creativity, innovations and readiness to take risk as well as ability to plan and manage projects seeking achievement of the set goals (Recommendation of the European Parliament and of the Council of 18 December 2006 on key competences for lifelong learning, 2006).The underlying knowledge, abilities and value orientations embrace ability to identify personal, professional and (or) business opportunities, including aspects of a broader attitude that reveal the context where people live and work.
In recent years entrepreneurship has been identified as a general competency or a competency, which is relevant to all the citizens and related to creativity, resourcefulness, new ideas and their implementation, therefore its education acquires a particular significance to implementation of social, technological and economic changes in various countries (Dudaitė & Žibėnienė, 2012, p. 166).Entrepreneurship as a general competency embraces two underlying aspects: 1) necessary knowledge, abilities and value orientations, whereof development can facilitate implementation of business ideas; 2) entrepreneurship is not linked only to economic activities and setting up of business but is associated with every member of contemporary society in all the spheres of his activity and life (Lackéus, 2015; Entrepreneurship Education at School in Europe, 2016, p. 21).Entrepreneurship competency consists of components, which include: 1) knowledge and abilities necessary for implementation of a specific activity; 2) abilities and value orientations linked with personal traits and behaviour (Komarkova, Gagliardi, Conrads, & Collado, 2015b, p. 35).
Generalising it can be stated that the concept of entrepreneurship is analysed in different contexts and is interpreted differently particularly because of the use of the concepts of entrepreneurship, entrepreneurship abilities and entrepreneurship competency.
The competence of entrepreneurship consists of knowledge of business, entrepreneurship skills, value orientations, personal traits and their comprehensive development, which also contributes to implementation of goals set for other study subjects.
In the programme Economics and Entrepreneurship in Lithuania, the content of education is provided in concentres.The content of each concentre consists of comparatively separate areas, which are integrated into the process of education: orientation in the market, business organisation and development of entrepreneurship skills, etc. (Pradinio ir pagrindinio ugdymo bendrosios programos, 2008).
The aim of teaching economics and development of entrepreneurship in a secondary school is, having selected the study subject of education, to develop knowledge and understanding of ongoing economic processes, to develop abilities to manage personal finances as well as to build up entrepreneurship competencies, to learn targetedly, to plan own life and career, to form a scientifically grounded attitude towards constantly changing economic activities (Vidurinio ugdymo bendrosios programos, 2011).
The structure of the programme Economics and Entrepreneurship consists of 5 content areas: 1. Orientation in the market, 2. Financial management, 3.Business organisation and development of entrepreneurship skills, 4. Analysis and evaluation of the state's role in economy and economic indicators, 5. Participation in international markets.
The further research was conducted seeking to establish the significance of integral parts (components) of the programme to competencies to be developed.Multiple Criteria Decision Making (MCDM) was applied evaluating the significance of the areas of the content mathematically processing and systemising data (Simanavičienė, 2011;Podvezko & Podviezko, 2014).Recently the method has been more and more widely used for evaluation of complex processes in social and economic phenomena.

The analysis of scholarly literature, strategic documents of the European Union and the Republic of Lithuania, General Education Curricular and the Study Programme of Economics and Entrepreneurship
Multiple criteria methods are applied when it is necessary to choose a rational alternative from a specific list of known alternatives, to systemise and mathematically process the presented situation -to form a line of alternative priorities (to rank alternatives), which showed an advantage of one alternative compared to others (Simanavičienė, 2011;Podvezko & Podviezko, 2014).
The number of multiple criteria is rather big and they differ in their complexity and specifics.However, none of them has been acknowledged universal for the significance of criteria.Scientists suggest using several methods for analysis of a complex phenomenon and studying of reasons for mismatch in evaluation.An indirect assessment method, i.e., ranking of 5 content areas in descending order of their significance (the most significant one is assigned the rank equal to 1, second most significant one is ascribed 2, etc.).The best variant is expressed by the lowest value of the ranks established by experts.A direct assessment method, when the sum of all the weights of all the assessed criteria provided by each expert is one (or 100 %) and the highest evaluation complies with the highest value, was employed (Podvezko, 2005(Podvezko, , 2006(Podvezko, , 2008;;Podvezko & Podviezko, 2014).
The data of expert questionnaire were processed and calculated using Microsoft Excel 2010.
To establish the level of agreement of experts' estimates of 5 areas of content, the Kendall's coefficient of concordance W (Kendall, 1962), which is based on the ranking of the evaluated criteria, was applied (Podvezko, 2005(Podvezko, , 2006 where r -the number of experts (rankers), m -the number of criteria being ranked, S -the sum of squared deviations of each rank from the mean.
The mean of sums of expert evaluations was calculated applying the formula The sum of ranks of each criterion i e is calculated following the formula   (5) The highest value of S is calculated according to the formula ) The closer the value of W is to the one, the higher concordance of expert evaluation is observed.If evaluations of experts significantly differ, the value of W is close to the zero (Podvezko, 2005, 2006).
The direct evaluation method was also used, when the sum of all the assessed criteria weights was evaluated in percentage, when the sum of evaluated criteria of each expert The mean values of each assessed criterion and their weights i  (in percentage) are calculated applying the formula ( Since evaluations of experts are based on their subjective opinion, they are frequently contradictory, therefore in multiple criteria evaluation it possible to rely on their evaluations, on their general averages, to be more precise, only if they are not contradictory, i.e. their concordance is proved (Podvezko, 2005(Podvezko, , 2006;;Podvezko & Podviezko, 2014).

Research ethics.
The research was carried out following fundamental values, universally acknowledged norms of ethics (Weiss, 2006;Kardelis, 2016;Ruškytė, 2016), following the principles of objectivity presenting results based on the research data; reliability conducting the research, describing research methods and expediency of their application to receive reliable data; honesty and respect for other researchers properly making references to scholarly and other sources and authors cited in the present research; goodwill and voluntarism that leave the right of self-determination of participating in the research; anonymity using the obtained data exclusively for the purposes of research with permission from all the participants.

Evaluation of significance of areas of content of the programme to development of school learners' entrepreneurship education.
The sub-chapter presents the generalised data acquired after multiple criteria evaluation of significance of content areas in the programme of general education to development of school learners' entrepreneurship expressed.
The data of expert evaluation are processed and their level of concordance is calculated applying the formulas from 1 to 8.
The data of evaluation of significance of 5 areas of content obtained applying the method of indirect assessment (ranking) are presented in Table 1 and Picture 1 and show that the most significant area, according to experts, was Business organisation and development of entrepreneurship skills, Financial management is ranked second, Orientation in the market -third, Analysis and evaluation of the state's role in economy and economic indicators is seen as the fourth, and Participation in international markets is ranked as least significant out of five areas.The sum of all the ranked components i e = 240 is calculated applying the formula (2), where m = 5 -the number of components, r = 16 -the number of experts.The sum of each ranked component is received according to the formula (3).The mean of all the sums of the ranked components e = 240/5 = 48 is calculated applying the formula (4).The sum of squares S = 1990 of the deviations from the total mean e is received using the formula (5).The highest value max S = 2560 is calculated applying the formula (6).The value of Kendall's coefficient of concordance W = 0.777 is calculated using the formula (1).The sum of weights of all the five ranked components i  = 1 are calculated applying the formula (7).The mean values and weights of each component i  (as a hundredth of means) are calculated applying the formula (8).

Table 1. The evaluation of significance of 5 content areas in the programme to development of school learners' entrepreneurship using the method of indirect assessment (ranking)
The data of evaluation of significance of content areas obtained applying the method of direct assessment (in percentage) are presented in Table 2 and Picture   The comparison of the data of expert evaluation conducted employing the methods of indirect assessment (ranking) and direct assessment (in percentage) reveals that evaluations in both cases coincide.
The processing of the data of expert evaluation showed that Business organisation and development of entrepreneurship skills (ranked 1st; 27.69 %; i  = 0.277) is the most significant content area in the programme of school learners' entrepreneurship education, Financial management (ranked 2nd; 21.56 %; i  = 0.216) is the second most important.
Participation in international markets (ranked 5th; 12.81 %; i  = 0,128) is evaluated as the least significant area.
The evaluations of experts may be considered reliable as the calculated value of the Kendall's coefficient of concordance W = 0.777 reveals a reliable concordance.

Discussion
In their works the scholars present a considerable number of models for entrepreneurship education but they are not targeted at general education schools and differ in their discussed structural components.Therefore, there emerges a question, which of the components are most significant to development of school learners' entrepreneurship and which of them should receive most attention in schools of general education.
As scientific literature does not provide an opinion regarding the use of methods that should be applied evaluating the significance of content areas of the programme to development of school learners' entrepreneurship, this issue is also an equally important problem.
The qualitative research methods are highly popular in scientific research, which is based on identifying attitudes towards entrepreneurship, education programmes, methods, etc., therefore their results are regarded as not objective and reliable (Weiss, 2006).The scholars, who follow positivist approach that acknowledges possibility of objective research, usually apply quantitative rather than qualitative research methods (Arthur, Waring, Coe & Hedges, 2012; Creswell, 2014; Kardelis, 2016).Carol H. Weiss, a professor of Harvard University, suggests quantitative methods for evaluation of programmes (Weiss, 2006, pp. 141-143, p. 266).
Seeking to get as objective and reliable data as possible, the significance of 5 content areas based on evaluations of experts, was identified applying multiple criteria.
The research results substantiate that Business organisation and development of entrepreneurship skills is the most significant content area of the programme of entrepreneurship education, whereas Financial management is ranked as the second one.Naturally such areas should receive the biggest attention.
Since the number of multiple criteria is rather big and they differ in their complexity and specifics, none of them has been acknowledged as universal.The opinion of experts, who carry out evaluation, is usually different or even contradictory.Therefore scholars suggest using several methods for analysis of a complex phenomenon and studying of reasons for mismatch in evaluation (Simanavičienė, 2011;Podvezko & Podviezko, 2014).Therefore the results may be different having chosen other multiple criteria methods, having changed the composition of experts or the number of rated components.
, which serve as basis for the presented concept of entrepreneurship competency in the context of general education.The questionnaire survey of experts was carried out seeking to establish the significance of areas of the content in the Study Programme of Economics and Entrepreneurship for development of school learners' entrepreneurship.Stages of expert survey organisation.The questionnaire form for experts was designed, which provided a possibility of evaluating significance of five areas of the content of the study programme to development of school learners' entrepreneurship in ranks and percentage expression.A group of 16 experts was selected on the basis of non-probability targeted sampling (considering competence, qualification and experience of experts in programme development, their working experience in economics and entrepreneurship): 8 developers of the programme Economics and Entrepreneurship (6 teachers methodologists from Lithuanian general education schools and Republican Association of Teachers of Economics, one representative from the Department of General Education and Vocational Training of the Ministry of Education and Science of the Republic of Lithuania and one representative from Public Institution Junior Achievement Lithuania), 8 independent experts (3 teachers from Lithuanian University of Educational Sciences, who train teachers according to the study programmes of Economics and Entrepreneurship, 2 representatives from Lithuanian Economic Association, one representative from each of Lithuanian Confederation of Industrialists, Lithuanian Business Employers' Confederation, Vilnius City and County Business Employers' Confederation, currently renamed into the Forum of Regional Development).The expert survey was conducted in December 2016.
The total mean value of these ranks e is calculated applying the formula the number of criteria).
number of experts,   -the evaluations of experts).

Fig. 1 .
Fig. 1.The evaluation of significance of 5 content areas in the programme to development of school learners' entrepreneurship using the method of indirect assessment (ranking) organisation and development of entrepreneurship… Analysis and evaluation of the state's role in economy and… Participation in international markets 2 and reveal that the most significant area is Business organisation and development of entrepreneurship skills (27.69 %, i  = 0.277), Financial management is ranked as second most relevant (21.56 %, i  = 0.216), Orientation in the market (19.19 %, i  = 0.192) is seen as the third most important, Analysis and evaluation of the state's role in economy and economic indicators (18.75 %, i  = 0.188) is indicated as the fourth most significant area,whereas Participation in international markets (12.81 %, i  = 0.128) as ranked as the least important area out of five.

Fig. 2 .
Fig. 2. The evaluation of significance of 5 content areas in the programme to development of school learners' entrepreneurship using the method of direct assessment (in percentage) Financial managementBusiness organisation and development of entrepreneurship skillsAnalysis and evaluation of the state's role in economy and economic