1 Introduction
Governments around the world have expressed concern about the low level of financial literacy among their citizens. In this context, in 2008, the OECD created an International Network on Financial Education (INFE) to facilitate the sharing of experience and expertise among international experts from public bodies on the topic and to promote the development of both analytical work and policy recommendations.
The work of the OECD/INFE focuses on financial literacy and education policy areas considered to be particularly topical. Among these, measurement of the level of financial literacy at both national and international levels rapidly emerged as critical in order to better inform policy-makers' interventions.
Measuring the level of knowledge and understanding of a population and assessing its behaviours with regard to finances are fundamental to identifying potential needs and gaps relating to financial literacy, as well as to identifying at-risk groups. Once a benchmark is set, the measurement exercise can be repeated in order to evaluate the impact of large-scale interventions aimed at addressing the issues.
The comparison of such a measure across countries will provide some important information about which national financial education and/or consumer protection practices work best, further guiding policy-makers' decision-making process. Harmonised data from across a wide range of countries is also of huge interest to researchers, academics, and policy analysts seeking to gain a deeper understanding of the variation in levels of financial literacy around the world.
The lack of such an international measure and comparable data, along with requests by many countries for good practices on obtaining a robust measure of financial literacy at the national level, prompted the OECD and its INFE to launch projects aimed at filling these gaps and meeting these requests. Two projects have been initiated in recent years in this area. The first is the development and implementation of an internationally comparable approach to assessing the financial literacy levels of adults; the second is the measurement of the financial literacy levels of 15-year-old students via the 2012 OECD Programme for International Student Assessment (PISA).
This paper focuses on the first of these projects. It discusses the development of the OECD/INFE international survey on financial literacy, highlighting the related challenges, the process of identifying suitable questions, and the methodology used to collect data by countries piloting the survey instrument (Section 2). It then introduces the countries participating in the pilot (Section 3). Finally, while the data have only recently been collected, the paper also discusses how the data will be analysed and the next steps for the project (Section 4 and conclusion).
The paper is relevant to this special issue of the Journal of Pension Economics and Finance, as it discusses a survey instrument that will enable further and more detailed international comparisons of how much people know about financial matters (i.e., levels of financial literacy), who knows the least, and whether this varies by country. While the questionnaire is primarily designed to measure financial literacy, the measure created will also enable secondary analysis to consider the extent to which financial literacy appears to matter across a range of outcomes – in particular relating to the use of financial products and, to some extent, planning for retirement.
2 Developing the OECD/INFE international survey
2.1 Difficulties in undertaking an international comparison
Huston (Reference Huston2010) identified three obstacles to creating a standard measure of financial literacy. The first is the lack of a common construct, the second is the lack of a comprehensive set of questions to measure all components of financial literacy, and the third is the lack of guidance in interpreting the measure created.
In our experience, consistency should be added to the list compiled by Hudson. Indeed consistency and the need for a common construct appear to pose the biggest challenges to an international comparison. Lack of consistency can occur at various stages, including during the translation and interpretation of questions, the sampling process, and the data coding and subsequent analysis.
These issues were addressed during the development phase of the pilot study, as discussed in the next section. The lessons learned through this pilot will enable further modifications for the final questionnaire and for guidance on best possible methods of data collection and analysis.
2.2 Agreeing on a common construct
As noted above, the first challenge faced in developing a measure of financial literacy is in establishing agreement about what this measure should consist of. In the framework of the INFE and for the purpose of the survey, financial literacy has been defined as ‘a combination of awareness, knowledge, skills, attitude, and behaviours necessary to make sound financial decisions and ultimately achieve individual financial wellbeing’. This working definition reflects the widely acknowledged OECD definition of financial educationFootnote 1 and makes clear that financial literacy includes knowledge but also goes further to include attitudes, behaviours, and skills. It stresses the importance of decision-making – applying knowledge and skills to real life processes – and it indicates that the impact should be improved financial wellbeing.
The definition of financial literacy describes a complex phenomenon that cannot be directly measured: there is no single question that can be administered to identify whether someone is financially literate or not, which means that it is necessary to create a comprehensive set of questions that enable analysts to identify underlying traits. This set of questions can directly test levels of knowledge, while also exploring attitudes and asking about behaviours.
2.3 Creating a comprehensive set of questions
With a working definition in place, the process of developing the OECD financial literacy survey began by taking stock of 26 existing surveys across 18 countries that have questions that test financial knowledge and/or that ask about financial attitudes and financial behaviours (Kempson, Reference Kempson2010). This review was used to identify questions meeting the following criteria:
• questions that have been tested and proven to be of high quality and unbiased;
• questions that have already been used in national surveys; and
• questions that are correlated with the concept being measured (financial literacy).
Questions that met the first two criteria were assessed in order to identify those that would be most useful. A systematic approach was applied to identifying questions that correlate with the concept being measured, drawing particularly on the work of the UK Financial Services Authority (FSA). The UK FSA has undertaken in-depth development work to identify the range of areas in which people need to apply financial literacy (Kempson et al., Reference Kempson, Collard and Moore2005). They identified four such areas, the descriptions of which were fine-tuned following data collection and analysis (Atkinson et al., Reference Atkinson, McKay, Collard and Kempson2007): money management, planning ahead (financial planning), choosing financial products, and staying informed about financial mattersFootnote 2. Our OECD/INFE review exercise subsequently considered the extent to which questions asked in surveys undertaken elsewhere could be classified according to these areas. Consideration was also given to whether there were some other aspects of financial literacy that were deemed important but that fell outside of these areas.
The UK FSA areas appeared to adequately reflect the range of questions asked in national surveys. The review also showed that some surveys incorporate questions designed to pick up additional explanatory factors such as psychological biases.
Once questions had been categorised according to which areas they belonged to, it became possible to identify themes and even questions that were common to a number of surveys. These frequently used questions were then reviewed by external experts, and following a process of edits and amendments, a series of 19 core questions were chosen to form the basis of the pilot questionnaireFootnote 3.
The core questions include eight financial knowledge questions. These vary in complexity and reflect the range of concepts that are assessed via other surveys, including interest, inflation, and diversification.
In order to strike a balance between overly complex and overly simple knowledge questions, fairly simple concepts in our core questionnaire are assessed via open-ended questions, while more complex questions are assessed with multiple choice questions.
These knowledge questions were specifically chosen to be as close as possible to questions used widely in studies linking financial literacy to specific financial outcomes (see, e.g., Hilgert et al., Reference Hilgert, Hogarth and Beverly2003; van Rooij et al., Reference Van Rooij, Lusardi and Alessie2011; Lusardi and Mitchell, Reference Lusardi and Mitchell2008). In some cases, they were amended on the recommendation of experts or to meet the requirements of an international survey (see Fowler, Reference Fowler1995, for a detailed discussion of best practices in designing survey questions).
The OECD questions addressing knowledge of interest rate calculation, inflation, and risk diversification are similar to the three questions used in the Financial Literacy around the World (FLaT World) project. The full set of knowledge questions used in the FLaT World project are also incorporated in our recommended list of supplementary questionsFootnote 4 for an even wider international comparison on financial literacy.
The 11 remaining questions in the core questionnaire capture behaviour and attitudes, focusing on financial control and making ends meet, choosing and holding financial products, and financial planning.
All questions that mention financial products are written using generic terms (such as ‘unsecured loan’) that countries can edit to allow their questions to be context specific. There are questions asking about awareness and use of a wide range of financial products that will provide a useful indication of financial inclusion in the surveyed countries. The same questions can also be used to indicate a variety of other outcomes that may be expected to be related to financial literacy, including retirement preparedness, decisions to invest, and use of credit. The core questionnaire can be supplemented with additional questions to provide a greater depth of knowledge about the surveyed population. In order to facilitate such a process, the OECD/INFE Supplementary Question set includes questions on topics such as respondent satisfaction with their current financial condition, factors they think should be taken into account when planning for retirement, whether they believe that their retirement plans are sufficient, and whether they are informed about their pension plan.
2.4 Collecting the data
The 19 core financial literacy questions and associated socio-demographic questions have been combined into a survey instrument (the ‘core questionnaire’) that is designed to be read out loud by an interviewer. The questions are grouped by topic, and each topic progresses logically (e.g., there is a question that asks whether the respondent has ever been unable to make ends meet before a question asking what they did in such a situation). Care has been taken to ensure that the questionnaire does not start with sensitive questions that may dissuade participants from continuing. Guidance has also been provided to the interviewers about the importance of the survey and the reason for asking the questions, in order to reduce the chance that the interviewer becomes uncomfortable asking potentially sensitive questions.
The survey is intended to cover individuals aged 18 and above. A minimum of 1,000 interviews should be achieved, using face-to-face or telephone interviews.
The decision to interview individuals regardless of whether they are the main household decision-maker or self-identified head of household reflects the generally held belief that financial literacy is a skill that every person needs in order to function in today's financially orientated world. The minimum age of 18 reflects the age at which it is possible to enter into contractual agreements, including credit agreements, in many countries.
The decision about the final approach to sampling has been left to each participating country in order to provide sufficient flexibility within the country, but instructions have been given to provide weights in order to make the data representative in terms of number of individuals (this may be necessary if only one person per household is picked regardless of the size of the household), gender, and age. Participating countries are also asked to consider whether it is necessary to weight the data to be regionally representative, and whether they should oversample particular groups in order to undertake additional analysis at the national level.
2.5 Interpreting the data
With the wealth of data made available from the core questionnaire, it will be possible to use the data in various ways. First, the results of the financial literacy questions can be analysed in order to assess levels of financial knowledge across countries. The OECD/INFE questions that closely resemble the questions used in other papers in this special edition of the Journal of Pension Economics and Finance are included in Table 1; there are also other questions that can be combined with these to create a financial knowledge score. Second, these results can be combined with other data to create a comprehensive measure of financial literacy.
Table 1. Knowledge questions
In all cases, interviewers were also instructed to note if the respondent said they did not know or refused to answer the question.
For our final measure of financial literacy, we will incorporate the knowledge score into a wider indicator that includes attitudes and behaviours, as indicated by our working definition. The comparison of financial knowledge, attitudes, and behaviour will also provide relevant and unique insights into the relationship between these factors at an international level.
Our recommended approach to measuring financial literacy at an international level will be guided by the findings of our pilot exercise, described in Section 3.
3 Status of countries participating in the pilot exercise
The OECD/INFE has over 80 low-, middle-, and high-income member countries, of which 12 have recently piloted the questions contained within the core questionnaire. Eleven of these countries are following the personal interview approach recommended by the INFE (Armenia, Czech Republic, Estonia, Germany, Hungary, Ireland, Malaysia, Peru, Poland, UK, and South Africa), while one (Norway) is testing an alternative method, using an online self-completion survey.
The countries participating in this pilot exercise offer a balanced mix of stages of development, financial inclusionFootnote 5, and financial and economic systems as well as a very good geographical representation with countries from Latin America, Africa, Western and Eastern Europe, and Asia.
Some of the pilot countries (including Ireland and the UK) have measured levels of financial literacy across their population before, while others will use the pilot study to create a baseline measure. Similarly, some have developed national strategies to tackle low levels of financial literacy within the country, while others intend to use the pilot process to inform the development of such a strategy.
Most of the participating countries are combining the pilot questionnaire with other questions. For instance, the Czech Republic has incorporated the pilot into its national baseline survey on financial literacy, designed to inform the launch of its national strategy on financial education. Malaysia is including it as part of a follow-up study with a special focus on low-income groups and household indebtedness. The Malaysian financial system includes a substantial share of Islamic financial institutions and will provide an interesting point of comparison with other countries.
The fact that the pilot sample is broad and will include some focused country-specific information will allow for the development of a thorough global analysis of financial literacy in its broadest sense, with some projects concentrating on particular aspects of interest, such as retirement planning or credit use. We discuss this further below.
4 Using the results of the pilot
The results of the pilot will inform the design of a final version of the core questionnaire as well as provide information about the levels of financial literacy within and across countries.
In order to enable policy-makers to make the most of the data from this pilot study, the national responses will be analysed in various ways. Answers to many of the questions will result in very interesting information about the behaviour, attitudes, and knowledge of individuals and groups of people within a country. Responses to these questions will therefore be used to describe and compare particular aspects of financial literacy across groups within and across countries.
We will use the pilot data to develop a guide for future waves of data collection as well as to obtain an overall measure of financial literacy. As discussed above, this overall measure will incorporate financial knowledge behaviour and attitudes.
Once we have a measure of financial literacy, regression analyses can tell us whether there are socio-demographic factors that predict certain scores. This will be of use to policy-makers looking for ways to identify groups of people most in need of financial education or most at risk from a lack of consumer protection policies. For example, it will be very useful to INFE members and researchers who are interested in the lower levels of financial literacy among women, as demonstrated in a number of studies (see, e.g., Lusardi and Mitchell, Reference Lusardi and Mitchell2008) and documented in many of the papers in this special issue of Journal of Pension Economics and Finance.
The scores can also be used to help us explore variations in outcome variables in the data. For example, it will be possible to undertake analysis to assess whether higher levels of financial knowledge are associated with certain outcomes such as holding savings accounts (or simply saving), having a private pension or being unable to make ends meet.
Apart from this global analysis, the data are expected to be used for the development of various special projects at the OECD, including one on empowering women through financial education and another on the role of financial education in financial inclusion.
One of the additional benefits of using questions that have been tested in countries around the world is that it will be possible to compare findings over time. The UK and Ireland, for example, asked similar questions several years ago, so we will be able to compare those responses to the ones acquired during this survey.
5 Conclusions and next steps
It is illuminating to read a recent paper by Jappelli (Reference Jappelli2010) that states:
In principle, to enable cross country comparison, a single questionnaire should be administered to a random sample of the population in each country. … However, such an approach (like the PISA test of educational achievement amongst 15-year olds) would require substantial resources and coordination efforts.
It is clear from this statement that the OECD decision to measure financial literacy among adults and children is considered to be highly ambitious, but it also falls within accepted best practices in the field of international comparisons. Preliminary results of this global pilot exercise to measure financial literacy among adults are under way and are expected to be published in 2012. This study and data will provide a first set of international indicators on the level of financial knowledge, behaviours, and attitude of individuals in different settings.
The pilot exercise will also inform amendments to the core questionsFootnote 6, which are expected to become an international standard for financial literacy assessments. Countries that have not participated in the pilot exercise will be encouraged to use the core questions as a standalone product or as part of their endeavours to set national benchmarks on financial literacy. This will permit us to increase the set of internationally comparable data and thereby improve research capacity on financial education, allowing for the development of more robust and impactful international policy guidance. Repeating the collection of data across countries will also be on the agenda in the future.
