Understanding corruption risk assessment

Corruption risk assessment (CRA) is a methodology (that is, a set of guidelines on how to collect, analyse, and interpret data) – often accompanied by a tool (for example, a data collection instrument such as a questionnaire or data analysis tool such as a dashboard). It is used by various state agencies, private sector actors, and civil society organisations (CSOs) to monitor corruption risks in the public sector (within ministries, localities, public procurement, etc), as well as in private sector bodies. More specifically, a CRA ‘seeks to identify weaknesses within a system which may present opportunities for corruption to occur’.b6af76839708 This paper focuses only on the use of CRAs by public bodies, such as anti-corruption agencies or supreme audit institutions, which have a mandate to monitor corruption risks predominantly in the public sector. In addition, it is limited to well-established methodologies that have been implemented over several years; nevertheless, whenever possible, we also refer to emerging approaches. Different models for rolling out CRAs, their limitations, strengths, and whether or not they are fit for purpose are also assessed. While not heralding one approach as the best, this paper showcases possible options that may be more or less advantageous depending on available resources, political constraints, data quality, and other factors.

There are different understandings among public bodies of what a CRA involves and how to differentiate it from other related activities. First, some agencies differentiate integrity assessment from corruption assessment, while others use the terms interchangeably (eg, the UN Office on Drugs and Crime (UNODC) and Georgia’s Ministry of Justice). In the former case, integrity assessment is understood as a framework helping to combat corruption by assessing personal characteristics including ethical and moral principles and factors contributing to their formation.658d206da27f This paper predominantly focuses on corruption rather than integrity assessments.

Second, a further dimension through which CRAs can be distinguished is the inclusion of preventive mechanisms. While the literature on corruption risk methodologies shows an increasing interest in preventive mechanisms,68656edc04a9 this is not necessarily reflected in the ongoing practical implementation of CRAs. By implication, CRAs are distinct from assessments that are part of corruption risk management approaches.c1b55a557207 Corruption risk management involves addressing corruption risks through various administrative, financial, and legislative means.e88a84f5a645 Corruption risk assessment methodologies are mainly and ‘only’ aimed at discovering the causes and conditions for corruption to occur. Yet they can also inform policymakers or the public about preconditions, consequences, and particular cases of corruption,b93dfcf5df00 as well as influence the initiation of preventive actions. This study focuses on the latter approach, stand-alone CRAs.

Running a corruption risk assessment is also quite different from identifying corruption itself.8d190cd31c2a CRAs target potential weaknesses in institutional design, legislation, procedures, or regulations that can lead to corruption or are generally associated with corrupt behaviour. For instance, a single bidding indicator in public procurement does not necessarily identify that an exchange between buyer and supplier is corrupt. Yet for tenders receiving only one submitted bid in an otherwise competitive market, corruption is more likely to be found. For example, the tender requirements may have been tailored in such a way that only one bidder could fulfil the requirements, hence only one submitted a bid.dbd3520f5f34 Therefore, while some of the results of CRAs can lead to criminal investigations, in most cases they are instead used by public bodies to identify signals of systemic weaknesses where loopholes can be closed. Where CRAs reveal actual cases of corruption, the initiation of a criminal prosecution depends on the mandate of the assessing body and its deliberate choice.

The definition of corruption, as well as corruption risks, can significantly vary from one state agency to another. Therefore, rather than predefining those concepts, one of the methodological foci of this paper falls on understanding how agencies define CRAs and targeted corruption types and risks. Nevertheless, in most of the cases discussed in this paper, the corruption type targeted by public bodies does not focus on an assessment of individuals’ behaviour but rather on structural loopholes and favourable conditions for corruption to occur.9885e789d975

Usually, public bodies responsible for CRAs are either independent anti-corruption commissions or departments and divisions within other controlling bodies (such as, a court of auditors, ministry of interior, etc). While some risk assessment approaches may be applied in an unstructured and unstandardised way, many public bodies have started introducing more systematic approaches based on a standardised methodology. Such a methodology can rely on existing studies and international best practice, or on internal practices, self-assessments, and the general experience of the assessors. The application of CRAs can either lead to follow-on investigations and prosecutions or may be triggered by criminal cases, while they can also support preventive measures aimed at addressing risks before corruption actually occurs.

The methodology used in this paper includes desk research on selected cases and semi-structured interviews with representatives of public bodies, international organisations, and CSOs dealing with CRAs in the selected countries. The interviews were conducted with members of civil society and anti-corruption bodies in the selected countries who were either directly in charge of CRAs or experienced in it. In total, ten interviews were conducted (some with multiple participants from the same agency or organisation). Among the secondary sources0db61b296ad6 used for the desk research are various policy reports published by international organisations (eg, the Organisation for Economic Co-operation and Development (OECD) integrity reports) and civil society, as well as internal documentation of CRA methodologies provided by the responsible public organisations.

Corruption risk assessment methodologies around the world

This section offers a broad overview of different CRAs currently in use around the world. It first identifies three main types of CRAs based on the public organisation(s) managing the assessment. This categorisation has wide-ranging ramifications for methodology, resources, comparability of results, and follow-up mechanisms. The section then outlines further dimensions according to which CRAs can be classified, including data used and the types of corrupt activities assessed.

Three main types of CRAs

CRAs can be grouped into three broad groups, depending on which institution has the most agency in the analysis of the gathered information: centralised, decentralised, and transparency-oriented approaches. The first type of CRA, a centralised assessment, is carried out systemically by an external body, such as an anti-corruption agency, over a distinct time period. It is conducted due to a request or because the public body is subject to cases of criminal corruption cases and hence a general risk assessment is mandated. Such centralised assessments can be done by collecting data on the operations of the public body in question and identifying ‘red flags’ that signal corrupt behaviour (eg, in public procurement), legal assessments, or analyses of procedural compliance. Centralised, systematic assessments usually require significant resources, both administrative and financial. For instance, a systematic assessment of all public organisations over a predefined period requires many assessors. The standardised collection of quantitative data also requires that assessors have technical skills and the ability to properly store and analyse the data. One of the advantages of an external review is that it facilitates an unbiased look at internal processes, and with a proper follow-up mechanism (to monitor implementation of recommendations), it can serve as a useful instrument to prevent corruption. Yet, this methodology potentially adds to the bureaucratic burden of the public body being assessed. For instance, responding to a request from an assessing body to share all documents describing procedures in the organisation, as well as the main official documents issued by the organisation, involves a significant amount of work on the part of the organisation being assessed. At the same time, if the responsibility to collect (or at least prepare) the data lies with the assessed organisation, the anti-corruption body does not need significant resources for data collection.

One example of a centralised, systematic CRA carried out by an external body is the Slovakian Corruption Prevention Department’s (CDP) online tool from 2020. This aims at collecting information from ministries through an electronic questionnaire,d7b2dce93b9a which is part of the implementation of an Anti-Bribery Management System.e5914b627aec Once a ministry completes the survey, the CDP provides it with the results and recommendations on risk mitigation measures. In addition, the CDP uses diverse external sources of information (citizen complaints, records from an anti-corruption hotline, non-governmental organisation (NGO) and civil society reports, media investigations) to analyse corruption risks in various sectors of the economy and incorporate recommendations and results of the analysis into the national anti-corruption programme.008cd8465efd

The second type of CRA methodology can be characterised as a decentralised approach. This means that the assessed public organisation can either apply the assessment internally without any external guidance, or may use external guidance to some extent but delegate the assessment of corruption risks to internal staff. Such CRAs usually take the form of questionnaires, focus groups, or interviews and can either involve all employees of the body being assessed, or just those who hold relevant positions (human resources (HR), the budget office, mid-level managers, etc). These assessments often require less (administrative and financial) resources than a centralised, systematic methodology. Moreover, as they are decentralised, they allow participants to reflect on their individual experiences with corruption risks rather than simply following regulations.

However, the lack of external supervision might lead to a situation where employees cannot express their views freely but state what they assume top management would want to hear (that is, if employees consider top management to be part of a corruption risk issue). Furthermore, without additional incentives for management to actually act on the concerns raised by the assessment, the effectiveness of decentralised CRAs is questionable, as the question remains ‘who guards the guardians?’. 

For instance, in North Macedonia, the Ministry of Finance developed a decentralised CRA tool, namely a questionnaire for self-assessment for use by public bodies.271ca596408a The questionnaire was first introduced in 2018. Questions asked to the staff mostly relate to regulations (whether they are in place, how they are implemented), employees’ qualifications and professional skills, as well as anti-corruption activities undertaken by the body. This self-assessment can either be completed by all employees or by representatives of departments and management.a3bcf10271d0 The tool specifies that there are no wrong answers and no punishment, and the respondents are asked to be thorough and honest.

The third type of CRA is the transparency-oriented approach, which provides a source of information on corruption risks to the general public or simply serves to increase transparency in certain sectors, territories, and/or state agencies. Such methodologies usually target a particular type of public function (eg, state subsidies, public procurement, political parties’ donations, etc.) or provide indicators for certain territories (cities, municipalities, regions) or sectors (eg, education, healthcare, construction, agriculture). Often, a publicly accessible analytics website or mobile application depicts the data and corruption risk indicators in a user-friendly format.

Such tools are usually introduced by anti-corruption bodies or other ministries, as well as NGOs. The main advantage of this type of CRA is the transparency of data and their availability to the public, which brings the data to the attention of civil society, the media, or anyone interested in anti-corruption initiatives. At the same time, such CRAs seldom have any follow-up mechanism embedded to ensure alleviation of corruption risks. In addition, they require extensive resources (for the collection of data, design, website/app hosting, systematic updates, etc.).

An example of such a CRA methodology can be found in Slovenia. A website, named ‘Erar’, makes available data on the use of public money in the country.2d52dd851275 The data behind the website are collected and maintained by the Commission for the Prevention of Corruption, and combined with data from the Ministry of Finance, various official registers, and the Office for Prevention of Money Laundering, among others. The main goal is to provide access for the media, civil society, and the public to review the spending of state agencies and state-owned companies. Corruption risk indicators are not provided, but rather there is a list of transactions, gifts, business restrictions, and the like; thus users have the possibility to identify potential risks by themselves and to use these data for investigative purposes.

Public body conducting the CRA

In many countries, the three types of CRAs detailed above are conducted by some sort of anti-corruption body, which can be either a stand-alone independent body or may be a subdivision of other ministries (the ministry of justice, ministry of finance, etc). Other countries develop systems of cooperation between different agencies and public bodies to conduct risk analyses in coordination. For instance, in Austria, a CRA aims at preventing money laundering and the financing of terrorism.4d83c5862a3f It depends on cooperation between the Federal Ministries of Finance, Justice, the Interior, Digital and Economic Affairs, and European and Foreign Affairs, as well as the Financial Market Authority (FMA) and the Oesterreichische Nationalbank.90ca80327dc4

Data used in the CRA

CRAs can be categorised depending on the type of data they use. There are two major data types involved in risk analysis: primary and secondary, within which anti-corruption bodies can focus on quantitative or qualitative data. Primary data sources include key informant interviews, focus groups, or statistical data collected by the implementing anti-corruption agency. The Indonesian Corruption Eradication Commission (KPK) serves as an example of an agency using various sources of primary data for CRAs and combining various methods to process the data collected (Box 1). Besides regulatory analysis, the KPK conducts surveys to assess the performance of public services, anti-corruption initiatives, and public surveys of people’s perceptions of corruption.ea6714b18ed9

While most CRAs rely on primary data, secondary sources can also be used to either prioritise audits by identifying the highest-risk agencies or for the audit itself. For instance, the Lithuanian Corruption Prevention Department monitors media investigations and the corruption perception surveys of various ministries to identify which public bodies to audit. CRAs in Italy similarly rely on data collected by the National Statistical Service on various socio-economic indicators of territories.

Corruption types analysed by the CRA

CRAs can be differentiated through the scope of assessment, that is, by the types of corrupt activities they explore. While some CRAs target highly specific types of corruption risks (eg, in legal regulations), others attempt to create a multifaceted risk assessment system involving more complex indicators that analyse different sides of corruption risks. For example, countries like Serbia, Moldova, Montenegro, or Albania focus solely on corruption loopholes in legislative frameworks and regulatory coherence and transparency.67715bbaeddb Other examples, like South Korea, enhance legislative analysis with integrity assessment surveys, self-assessments of anti-corruption efforts, focus groups, and public surveys (Box 2).441e602ea9f4 Naturally, more comprehensive systems require significantly more resources (for example, a public survey alone for the South Korean Anti-Corruption and Civil Rights Commission costs around US$2,500,000). As the case studies will show, some countries’ anti-corruption bodies have sufficient resources at their disposal to implement complex multilevel CRAs, while others limit their analysis by applying narrow tools.

Four diverse case studies

This section is devoted to four case studies of different CRAs in the Netherlands, Lithuania, Italy, and Mexico. The selection of cases was driven by three motives: (1) to present CRAs within a variety of institutional frameworks and with different challenges; (2) to cover both comprehensive large-scale methodologies (using a combination of different tools and covering a wide scope of corruption) and more narrow ones; and (3) to only include examples of CRAs that have been sustained and repeated over time. The Italian and Mexican cases provide examples of complex multilevel systems targeting various types of corruption practices in different ways. While the Italian system is very much data driven and systematic, the Mexican CRA can serve as an example of a system-in-progress experiencing significant obstacles due to weak political institutions, absence of proper cooperation among various ministries, as well as a lack of support from the current president and ruling party. By contrast, the Lithuanian and Dutch CRAs are much narrower and require less administrative and financial resources, as they target highly specific types of corruption. At the same time, the Lithuanian and Dutch CRAs show long-lasting results and improvements in institutional and normative frameworks,a23fe5710a81 while the Italian and Mexican systems are yet to demonstrate outcomes in the long run.

The Netherlands

In the early 1990s, the Netherlands’ General Intelligence Secret Service started developing a CRA methodology. It was to be used by various ministries to identify existing corruption risks within the organisations, as well as to suggest mitigation measures. The methodology was updated in 2003 and initially took the form of a manual for organisations to conduct a self-assessment, prepared by the National Integrity Office (BIOS, part of the Ministry of Interior). It was later updated and adjusted, introducing moderators and external guidance to ensure the provision of assistance. The updated system was called ‘SAINT’ with the goal to assess corruption risks in existing governmental agencies. In 2016, the state financing of the National Integrity Office was halted, and SAINT ceased to exist. The Whistleblowers Authority (WAA) was established in its place, with fewer staff members and a broader mandate. Nevertheless, the experience from the SAINT system was partially inherited and built on.350b3c7eeb45 The main difference between the two organisations and their mandate was, first, the scope of analysis: while the National Integrity Office focused on the public sector, the Whistleblowers Authority’s scope covers both the private and public sectors (including hybrid ones such as in the healthcare sector). Second, while the SAINT system aimed at corruption risk assessment, the WAA assists organisations and state agencies in integrity management.

Currently, the WAA mainly advises employers on the implementation of internal corruption reporting mechanisms (the prevention unit), as well as providing legal support for potential whistleblowers (the advice unit) and investigating reported allegations (the investigative unit). Results from the services provided are confidential and not disclosed to the public, yet the WAA is obliged to provide an annual report to parliament. In 2017 and 2018, the WAA commissioned surveys on internal reporting systems among confidential integrity advisers within organisations with more than 50 employees.bba1a29cee0b The surveys were conducted electronically with 50 closed questions and a sample of more than 300 respondents (however, the response rate was only around 50%). The results showed that in most organisations, internal integrity systems were in place, with around half of them being introduced with recommendations and guidance from the WAA.6a984692957d

It is planned that the SAINT system will re-emerge in the form of ‘IntoSAINT’, a self-assessment tool enabling supreme audit institutions (SAIs) globally to assess their vulnerabilities and monitor integrity violations.391834ad1d95 It has already been implemented in some countries. The project is supported by the Dutch Capacity Building Committee and has been chaired by the Mexican SAI since 2017, taking over from the Dutch Court of Auditors. IntoSAINT has been customised based on the SAINT system to address the needs of SAIs specifically. Among current certified IntoSAINT moderators are SAIs from South Africa, Tunisia, Indonesia, Cameroon, the Netherlands, Mexico, Fiji, and the European Court of Auditors.f3c6cca6b64f

SAINT system

Methodology description

The now-defunct SAINT system provided analyses conducted by the National Integrity Office (BIOS) concerning all kinds of public bodies (eg, police offices, municipalities, border authorities, ministries, provinces, education, prisons, etc). Its identification of which ministries and agencies to assess was based on voluntary requests from the organisations themselves. The public bodies contacted the National Integrity Office asking to be assessed (at the same time, every public organisation was obliged to conduct a CRA according to the legislation at the time). The philosophy of the system can be described as ‘making an assessment for employees with employees’, therefore serving as a good example of a decentralised approach. It implied that all the information on the organisation’s performance was collected from the employees themselves in the form of ‘guided discussions’, rather than through external centralised assessment.

The guided discussions that served as the main source of information involved a so-called ‘decision room’ lasting for about a day, with two representatives from the National Integrity Office (one technical person and one moderator) and up to 15 representatives of the agency or department’s staff. The staff members were selected by the BIOS from all the employees based on the position and knowledge of a respondent (one representative of each department, a few middle-level managers, a trusted person). The questions to discuss were formulated to identify the processes that were most vulnerable to corruption within the organisation. First, the NIS requested a list of all the most important processes in the organisation. Next, the list was narrowed down to the riskiest processes in terms of integrity management, based on the opinion of people working in the organisation. Finally, the employees were asked to evaluate weaknesses, by identifying who was responsible for decision-making, what measures were already in place to mitigate the risks (and whether they worked), and how they could be improved. During the discussion, the staff members shared their experiences, thoughts, and ideas (in both oral and written form), sometimes with answers shared anonymously with other participants, sometimes in the form of an open discussion, and sometimes as individual answers. All the answers, as well as the main points of the open discussion, were collected and later presented in the report, enriched with international best practice and solutions coming from existing literature and studies.

Altogether the SAINT system had eight people working on various tasks: two people were involved in conducting the risk assessment, while others provided integrity manuals and trained integrity officers. The division was subsidised by the Ministry of Interior, but the assessed organisations also paid to be assessed. The approximate price for having a risk analysis conducted was around €2,000–2,500, which together with the subsidy covered preparatory work, holding the guided discussion, writing the report, and in some cases, presenting it to the management.

Follow-up mechanism

One or two weeks after the guided discussions, the NIS would deliver a report of corruption risks to the management of the organisation, including recommendations on mitigation measures. The responsibility to implement these suggestions lay mostly with management itself. Therefore, the mandate of the Ministry of Interior was to assess and recommend improvements, but without investigative or prosecuting power over the assessed public agencies. The primary targets were institutional design, regulations, and procedural aspects of the agency’s operations; hence, personal compliance was not a part of the analysis. Respondents in the present study specified that participants were specifically instructed during the group discussions that examples provided by them should not be about particular individuals but rather about corruption risks at the organisational level more generally. This CRA was only conducted once for some agencies, while for others it was an annual assessment. In some cases, the risk analysis was first conducted within one department and then extended to others or to the whole organisation.

The results were only available to the assessed organisations, with specific attention to the potential sensitivity of the data; other ministries or public agencies were not permitted to access the information. Similarly, the final reports were not published open access, although occasionally the NIS would analyse common trends observed in certain types of public agencies and share these with the public. As there was no ex-post evaluation of the implementation of recommendations, there are no data on how effective the system was.

Pros and cons

All in all, the SAINT system had its advantages as well as its limitations. First, the system was not resource intensive, but partially for that reason it could not be a comprehensive solution for the systemic evaluation of public institutions as the institutions were assessed on request only. Second, the audit component (including assessment of actual procedures) was missing from the decentralised approach, although it was very useful for uncovering the experience of organisations in the field. Third, the methodology did not focus on individual behaviour, although it could address institutional corruption risks as well as procedural risks based on staff experience. The true effect of the approach remains unclear due to the lack of follow-up mechanisms and documentation.

Lithuania

The Lithuanian CRA is conducted by the Corruption Risk Division of the Corruption Prevention Department of the Special Investigations Service of the Republic of Lithuania (STT). The STT deals with corruption prosecution, prevention, and anti-corruption awareness. The risk assessment is carried out by one division, but there is a possibility to initiate an investigation in coordination with other of STT’s divisions. Moreover, once there is a criminal prosecution against a public official, the Corruption Risk Division will follow up with a CRA of the corresponding agency to identify reasons behind the case. The CRA methodology was fully launched in Lithuania at the beginning of the 2000s. Since then, it has been modified with the addition of new risks to the list of indicators and by applying different weighting to the risks. The CRA was inspired by a similar one from Hong Kong.8f622526b09f

Anti-corruption assessment of the existing or draft legislation

The main goal of the assessment is to identify discrepancies and inconsistencies between existing regulations and the actual activities or operations of state agencies. The Corruption Risk Division (CRD) can assess risks, advise on improvements, and later monitor compliance with suggested recommendations. It can either advise on legislation and regulations, or on the procedures used within an organisation. The public bodies assessed by the CRD include all ministries and public entities (independent, state or municipal institutions, state or municipal companies, etc.) at all levels of government. Private sector companies are not targeted by this assessment.

The primary target of the regulatory CRA is to establish the level of general compliance with procedural operations within an organisation. Therefore, the CRD assesses both individual cases of compliance and general institutional settings. However, in the final report, names of individuals remain undisclosed due to European Union (EU) regulations on data protection. If an individual is identified as being non-compliant with the regulations, the case can be sent to another division for further investigation. The STT has a right to start investigations and proceed with prosecution, including of cases revealed during the CRA.

Methodology description

The methodology itself includes two steps: first, the Corruption Risk Division requests all the legal documents regulating operations from the assessed body (of any type and at any level of legislation). Second, the CRD requests information on the procedures taking place in the organisation. Such documents include information on staff activities, thus the CRD can identify discrepancies between what is required by the regulations and what types of activities are actually taking place. Additionally, the CRD can request the actual documents produced by the organisation and signed by its officials. For example, if a public procurement body is being assessed, the CRD can ask for tendering protocols, signed contracts, etc. External sources of information are also taken into consideration while conducting the CRA, such as information provided by criminal intelligence, analytical anti-corruption intelligence, data from social media and the press, official registers, and citizens’ complaints. Such external information is mostly used to identify ‘red flags’, which are then followed up in the analysis of documents received from the agency.

When it comes to analysing the compiled data, the CRD has formulated a register of risks that can be used to identify which discrepancies between regulations and procedures signal a potential corruption risk. However, in many instances, the CRA is conducted without a standardised method of analysis, but rather through personal evaluation of whether the issue uncovered should be considered a corruption risk or not. In most of the cases, the discrepancy is considered a corruption risk if there is a repetitive pattern of non-compliance in practice. At the same time, corruption risks are ranked by the CRD as ‘critical’ and ‘less critical’; depending on this status, the follow-up monitoring is then either more or less thorough. The distinction between high and low risks was first implemented in 2021 and is based on whether assessors consider there to be a direct risk of corruption, or rather a more general non-compliance problem.

According to the legislation, the CRA for each public body must be conducted every four to six years. Eight officers work in the division, with each of them finishing around three or four assessment reports per year (one officer is assigned per organisation). The date for the next assessment is usually planned at the time when the previous one is finalised. In general, the resources needed for the CRA are limited, mostly being employee salaries, as the assessment itself is not costly and does not require any particular investment (since it is primarily desk research). Moreover, the STT’s divisions cooperate with each other; therefore, officers from other divisions can potentially be involved.

Follow-up mechanism

As a result of the CRAs, the CRD produces a set of recommendations aimed at eliminating instances of non-compliance and shares this with the assessed public bodies, as well as with the public (all reports produced by CRD are available with open access). Within three months, the relevant public body is required to share updates on how it is planning to implement the recommendations, or which steps it has already taken to mitigate the identified risks. After 12 months, the public body must share final updates on all implemented changes. If it refuses to implement the suggestions, it either must propose an alternative solution or justify the reasons for ignoring the recommendations. If critical risks have been identified within the public body, yet no changes are made, the CRD includes this case in its National Anti-Corruption Agenda and Action Plan. This exposes the body to public and media attention.

Respondents from the CRD who were interviewed for this paper, reported that the assessed bodies fully implemented the suggested recommendations in around 50% of cases. Another 30% were in the process of implementation, while the remaining 20% either did not implement the recommendations at all or implemented them only partially. Since the establishment of the STT, people’s perceptions of corruption in Lithuania have significantly improved.8449a050bfc8 However, based on respondents’ opinions, the effectiveness of the system could be further boosted by increasing the number of staff working with the CRA, as well as the creation of additional incentives for organisations to implement recommendations.

Pros and cons

In summary, while the Lithuanian CRA has evidence of effectiveness, it has some drawbacks. First, the scale and scope of the assessment requires highly qualified personnel with considerable administrative capacities to assess institutions. A commitment to provide sufficient resources and legislative support for such bodies is therefore required from the government or parliament. Next, the consideration of legal and procedural aspects is very useful for the analysis of the operation of a public body. At the same time, having some contextual indicators could be beneficial for a higher-level perspective to better understand the factors and root causes shaping corruption risks. Finally, the methodology involves only the top-down format of auditing, without allowing organisations to engage in self-assessment. While this approach has many advantages, it may overlook certain organisational characteristics familiar to people on the ground.

Mexico

The Mexican anti-corruption system (Sistema Nacional Anticorrupción, SNA) provides a CRA example with a multilevel, decentralised institutional arrangement. The SNA was introduced in 2016596de8b41e70 with the main aim to bring together state agencies working on corruption cases, along with civil society, academia, and international organisations, to increase transparency and make monitoring of corruption within government more systematic. One of the core ideas behind the SNA was to ensure the participation of citizens and therefore introduce an element of direct social accountability into corruption prevention activities. Besides representatives of citizens (through a Citizen Participation Committee, CPC), the system includes the Ministry of Public Administration, Superior Audit of the Federation, Special Prosecutor Office, National Institute of Transparency, and the Federal Court of Administrative Justice. The CPC is elected through the Senate, which appoints, for a four-year term, five citizens with a background in combating or studying corruption. This group of citizens is charged with monitoring and supervising the activities of the SNA, with independent analysis conducted by each member separately during their term in office. The coordination within the SNA is provided by the Executive Secretariat (SESNA) through the development of various guidelines and agendas for the involved ministries.

Sistema Nacional Anticorrupción and Citizen Participation Committee

The SNA is supposed to process any corruption-related inquiries and provides and shares all the information among Mexico’s law enforcement agencies. While the CPC is supposed to supervise the process of information exchange, it does not have the authority to start an investigation independently. Therefore, the main power of the CPC lies in providing recommendations and conducting screening of governmental agencies. The recommendations and studies are drawn up according to suggestions of – and in collaboration with – civil society actors and academics working on anti-corruption, as well as with the help of international good practice. All public bodies in Mexico can be assessed by the SNA, as can all levels of government. Since Mexico is a federal state, the SNA exists at the federal and regional levels (with 32 subdivisions in total), reproducing the same organisational structure and thus assessing the governmental agencies of the corresponding level. Yet, many regional SNAs are not fully compliant with the requirements established by the federal SNA. In addition, depending on available regional resources, local SNAs significantly differ in budget and structure.

The SNA usually prioritises bodies or sectors for assessment through analysis of publicly accessible data, civil society reports, or because of the personal expertise of the CPC members. For instance, one of the current members of CPC shared that her expertise was related to corruption in public procurement; therefore, one of the main reports she was working on during her term at CPC was a comprehensive quantitative and qualitative analysis of corruption in this sector in Mexico. The main target of such analysis is usually personal compliance; that is, the identification of public bodies or individuals prone to corruption risks. Cooperation between various ministries within the system helps secure access to necessary information. However, this type of analysis conducted by individual members is usually an individual project and does not involve other people from the SNA directly.

When it comes to the use of data, both primary and secondary data are collected for the CRA. The reliability of secondary data is mainly assessed according to the general reputation of the source organisation or institution (for example, from official complaint channels like whistleblowing procedures). For instance, the SNA considers the National Service of Statistics and Geography to be a reliable source for quantitative data on various indicators. Similarly, civil society reports or academic articles are used as starting points for further investigation. Some of the CPC members mentioned that they were also collecting data manually for their individual projects; for example, scraping the public procurement portal in Mexico by importing it from html format to data sheets. However, the SNA has yet to develop a standardised data collection tool. The CRA usually identifies corruption risks based on knowledge of common corruption techniques in Mexico, as well as international good practice.

National Anti-corruption Internet Platform

Methodology description and follow-up mechanism

Since the introduction of the SNA, one of the main results of its activities is the National Anti-corruption Internet Platform,40093f90e465 which publishes all data collected on asset declarations, public figures participating in tendering procedures, sanctions against public servants, as well as various materials on corruption in general. The idea behind the platform is to increase transparency and provide information about public officials, existing sanctions against them or respective public agencies, descriptive statistics of public procurement contracts, and details of declared assets. While this information is public, the reports produced by CPC, as well as guidelines and agendas produced by SESNA, are not intended for public dissemination. The assessments can further be used for investigative purposes or to prepare a case for criminal charges. Ministries involved can also impose sanctions or assign training in the case of minor violations.

Moreover, some of the respondents mentioned that there were limitations regarding contacts with the media and disclosure of information for political reasons. The SNA was introduced under the presidency of Enrique Peña Nieto as a civil society-driven initiative (which gained political support at the time), but it faced a significant cut in resources after the presidential elections in 2019. The new president – Andrés Manuel López Obrador – prefers state-controlled anti-corruption initiatives over more inclusive ones with direct citizen participation.0ac2b2dbb6f9 While anti-corruption discourse remains a key topic in statements made by the new president (and by his party, which has a majority in the Mexican federal parliament), the SNA has lost its political support and protection.

Pros and cons

There are many limitations in terms of the effectiveness of the CRA as implemented by the Mexican SNA. First, the significant decrease in resources and absence of political support have diminished the administrative capacity of the system. Second, the lack of a clear and standardised methodology guiding data collection and data analysis limits the ability of the SNA to perform comprehensive, indicator-based analyses. Nevertheless, the current National Anti-corruption Internet Platform can serve as an important source of information on public officials. Third, while the Citizen Participation Committee is supposed to supervise the SNA, guidelines on its activities as well as the powers of citizen representatives are limited and unclear. Finally, while one of the initial goals of the SNA was to create a system of checks and balances, reportedly collusion between law enforcement agencies and the ministries being assessed remains an issue, while there is inadequate control over conflicts of interest.

Italy

The CRA in Italy is conducted by the National Anti-corruption Authority (ANAC), which is an independent state agency established in 2012. ANAC’s structure consists of five board members appointed by the president and approved by the Council of Ministers for a six-year term. Altogether, there are 350 employees in the organisation. They work on preventive strategies, supervise the Three-Year Plan on Integrity within each state agency, collect data on public procurement, sanction non-compliance with integrity plans, manage whistleblowing complaints, and define anti-corruption and integrity standards and regulations. The Italian system of CRAs and corruption control involves comprehensive analysis of various contextual indicators as well as more direct methods. One of the primary foci of ANAC’s analysis is public procurement. ANAC has developed a tool known as ‘collaborative supervision’, which is used to identify risks and prevent corruption in public tendering processes. Additionally, since 2014, with the funding of National Operational Program Governance and Institutional Capacity, ANAC has developed and published openly a set of indicators identifying corruption risks in public bodies.

Measuring the risk of corruption at territorial levels and promoting transparency

Italy’s programme on risk assessment at the territorial levels (municipalities, provinces, and regions)a243fc15f898 was developed in cooperation with other national state agencies such as the Statistics Institute and Ministry of Justice. By collecting quantitative data at the local level, ANAC has developed a set of indicators through which the risks of corruption can be assessed. The system concerns local-level agencies (for example, local public procurement buyers) and presents a set of red flags that can be further used to initiate investigations, preventive measures, or may in general be scrutinised by civil society. The data are collected for all sectors to generate an overview of the local context more comprehensively.

Methodology description

There are three types of CRA indicators: contextual, procurement related, and municipal. The indicators of corruption risks, or ‘red flags’, have been developed in cooperation with civil society actors and academics working in the field of anti-corruption. The contextual indicators are calculated based on four dimensions: education (eg, percentage of the population with higher education); the presence of crime (percentage of various crime types in a locality, such as corruption offences, burglary, money laundering, etc); characteristics of the local economy; and the socio-economic development of the territory. Procurement-related indicators calculate the corruption risks in tenders at the provincial level. The indicators are divided by sectors, type (works, service, supplies), and year of publication of the tender. They are based both on economic factors (eg, quality/price ratio) and on administration of procedures (eg, ratio of non-open procedures, modified contracts, etc). Finally, the municipal indicators are identified through statistical analysis and observations of any correlation between potential corruption indicators and actual cases of corruption among local administrations. For instance, this set of indicators includes the extent of mafia infiltration, ratio of tenders held below the threshold, and size or complexity of the municipality.

The analysis does not directly point to individual risks of non-compliance, but rather helps to identify territories prone to higher corruption risks and brings public attention to the problem. Data were collected for the period 2014–2017 for each province in Italy. The assessment was funded by the National Operational Programme ‘Governance and Institutional Capacity 2014–2020’ – ERDF Fund. For each of the indicator types, a team of external experts comprising academics working in statistics, economics, and political science complemented the work carried out by the ANAC data experts. In total, between five and eight people worked collectively on each type of indicator. At first, the invited experts reviewed the most recent studies, describing methods of corruption measurement. This was followed up with discussions with ANAC representatives on the available data and their quality. Next, the team calculated the indicators and adjusted them based on the analysis conducted.

The collected data came from the National Statistical Service (Istat), National Database of Public Procurement (BDNCP), and other state bodies (the Ministry of Finance, Ministry of Justice). Before publishing an indicator, ANAC ensured there was statistical evidence to support the hypothesis behind it. For instance, the indicator for taxable income per capita, measuring the social well-being of the territory, was proved to negatively correlate with corruption by multiple studies. Some of the data are collected and systematised by ANAC itself. As ANAC inherited Italy’s public procurement platform, these data are maintained and collected by the Observatory within ANAC and then transferred onto the BDNCP. This then serves as the main source for procurement-related indicators.

Follow-up mechanism

The main goal of developing these indicators is to increase transparency and simplify access to information on corruption risks for the general public and civil society. Hence, all the collected data are published openly online in a dashboard format that allows filtering for year and province.6712efcffe19 The tool does not aim to initiate investigations or prosecution as it does not assess individuals. The published information can be used by the media, civil society, and businesses for the purpose of investigations, research, or business decisions.

Pros and cons

Based on the opinions of our interviewed respondents, the use of contextual, procurement-related, and municipal indicators to identify territories prone to higher corruption risks can prove useful and effective if properly advertised to the public and interested experts. The involvement of civil society actors and academics working in the field of anti-corruption helps to ensure the credibility of the indicators. Furthermore, the use of statistical evidence in the CRA to support the hypothesis behind each indicator enhances their reliability. Since the data were collected from various state bodies, this also helps to ensure the completeness of the data. However, the analysis does not point to individual compliance, which means the indicators may not be useful in identifying specific cases of corruption. Rather, they point to patterns of corruption risks. The reliance on statistical evidence may also overlook qualitative information that may be relevant to identifying specific corruption risks. Last, the need for regular updates of information requires resources be made available on an ongoing basis, which may not be sustainable in the long term. Despite these limitations, the use of indicators – as in the Italian CRA case – provides an effective means of identifying territories prone to corruption risks and raising public attention of the issue.

Public Procurement ‘collaborative supervision’

Another CRA tool developed by ANAC is a preventive mechanism to assess corruption risks in public procurement. As previously described, the agency collects public procurement data and collates them on the National Database of Public Contracts (BDNCP). Through this registry, any Italian authority can check whether a potential supplier company meets criteria for participation in specific tenders. The BDNCP also collects information from various other sources, including the tax register, Chambers of Commerce, criminal records, and others. The data have been collected from 2007 and cover more than 20 million public contracts. The information from the system is used by various ministries, including the Ministry of Finance, the Court of Auditors, and the Parliamentary Budget Office, as well as by different law enforcement agencies for investigative purposes.

Methodology description

The BDNCP system introduced a set of indicators in its open access portal, through which the user can assess various sectors, types of tenders, procedures, or specific public bodies or supplier companies. The BDNCP uses the same identifier for one company across multiple datasets, so it is possible to access full information about a supplier, openly, across datasets. The volume of data collected by ANAC has helped in developing a collaborative supervision model, which seeks to prevent corruption cases before they occur by alerting public bodies to potential risks and triggering investigation and preventive measures.

The collaborative supervision model was first introduced in 2015 and fully launched in 2019. The tool aims to assess procurement procedures upon the request of the contracting authority. The primary target of the assessment is legal compliance, as well as the correct execution of procedures. Within ANAC there are two entities responsible for the tool: the Office for Collaborative Approach and Special Surveillance (UVS) and the Special Operative Unit (UOS). The scope of tenders that can go through supervision by request of the contracting authority is limited to special events (eg, big sport or cultural events), tenders related to natural disasters, large infrastructure projects, or exceptionally high-value works, supplies, or services. In some cases, if a criminal incident or other form of misconduct appears in the tendering procedure, collaborative supervision can also be requested.

Follow-up mechanism

In order to request collaborative supervision, a public authority must submit all the documentation on the tender, including the appointed selection board, list of participants, estimated price, etc. ANAC also assists the procedure itself, which helps to forecast and prevent cases of corruption. After carrying out the checks, ANAC issues recommendations that can be used by the buyer to align the procedure. Usually, the checks should be made promptly (within a few days after the submission of the documents), especially when it comes to large infrastructure projects, to avoid delays in contract implementation.

The collaborative supervision model results are available for public bodies but are not published open access, as the main goal is to assist the buyer rather than to inform the public. Mitigation of corruption risks should be carried out by the public body, with corresponding recommendations from ANAC. Based on ANAC’s calculations, collaborative supervision has helped to save around 10–20% (or more than €900 million) annually in health sector procurement alone.606252bb5e4a

Pros and cons

The collaborative supervision model developed by ANAC offers many benefits for Italian authorities, including stopping corruption by alerting public bodies to risks and triggering investigation and preventive measures. Yet, for another agency trying to recreate such a methodology from scratch, collaborative supervision would require significant resources. ANAC has benefitted from public procurement data and regulations, so saving on the effort involved in establishing such a method. Collaborative supervision is mostly aimed at large-scale procurements, which are more likely to be associated with higher corruption risks. Yet, at the same time, such a CRA overlooks smaller-scale tenders and does not provide any assistance to small-scale buyers, leaving some procurement processes vulnerable to corruption risks. Finally, lack of open access to the results of collaborative supervision may limit transparency and accountability.

Main challenges and recommendations for CRAs

Based on our analysis of interviews and policy reports on CRAs, we identified four groups of challenges and limitations. First, CRAs and implementing public bodies face a range of challenges imposed by their broader political, institutional, and social environment, such as sufficient and sustained independence or adequate resource allocation. Second, data quality and availability impose fundamental limitations. As many anti-corruption bodies rely on data provided by other ministries or public agencies, they often face difficulties in either accessing the data or making sure the data are of high quality, standardised, and comparable with other datasets. Third, some CRAs have a limited focus by design and therefore can only assess specific types of corruption, leaving others out. Finally, ensuring a follow-up mechanism and monitoring framework for implementation is often challenging and not carried out comprehensively.

Challenge: Institutional constraints

Political and institutional constraints often limit the capacity of anti-corruption agencies to conduct effective CRAs and ensure assessment results impact anti-corruption practice, as can be seen in the case of Mexico. Collaboration across public institutions is crucial to support the CRA and its operations, among others, by giving access to data and key experts. Necessary support for CRAs must include adequate financial resources and administrative and analytical capacities, if the agency is to carry out the assessment and effectively support the fight against corruption. The mandate of the public body conducting the CRA and political constraints likely lead to the prioritisation of certain types of corruption while ignoring others (eg, focusing on bribery rather than high-level corruption). Therefore, it is essential to maintain a supportive framework of collaboration and political support to ensure CRAs are successful.

Recommendations

Substantial political and institutional support should confer adequate resources to the CRA itself and the public bodies participating in the assessment. Ideally, participation and cooperation in the CRA should be mandated by law, which ensures institutional continuity and sufficient assessment coverage. While voluntary assessments exist and operate, usually some legal leverage is needed to motivate the cooperation of assessed bodies and therefore to minimise the administrative burden on corruption prevention commissions to request and collect the necessary data. For example, one of the respondents mentioned that sometimes they had to spend a lot of time and effort getting documents from ministries.

Challenge: Data quality and accessibility

A key challenge raised by the interviewees was data quality (reliability, accuracy, completeness, and consistency) and data accessibility. First, some CRAs do not use primary data as they lack their own data collection tools (such as, a survey instrument) and personnel, which severely limits the scope and depth of the analysis. High-quality and detailed CRAs need reliable data providers with comparable data across time, organisations, and government activities. Guaranteeing these elements is only possible with standardised control over data collection and database definitions, which requires skill and resources. Challenges of data quality arise less frequently when the CRA relies on a national statistical office to collect data, which are often high level rather than large scale and granular. Second, data access itself is a fundamental challenge in many cases. Respondents stated that some assessed public bodies did not collect data in dataset format, but as numerous documents in pdf/doc/Excel formats. This necessitates information requests on a case-by-case basis, leading to delays in the assessment process and limiting comparisons across organisations.

Recommendations

While it is important to ensure that anti-corruption bodies have access to their own data collection tools and personnel, establishing cooperation with reliable data providers can alleviate resource limitations. Additionally, having access to comparable data from multiple sources can benefit the risk assessment. Instead of storing numerous separate documents in inconsistent formats, it is important for anti-corruption bodies to invest into dataset building. Furthermore, agencies should use the most suitable methodology given available data and consider their resources when developing an affordable methodology, finding the balance between extensive analysis and focused assessment of targeted corruption types.

Challenge: Methodology and analytical focus

While some CRAs aim at a comprehensive, system-wide assessment, others conduct a narrow analysis targeting a specific type of corruption. The latter approach has its advantages, as it allows for a clear and comparable assessment focus without scattering attention on multiple issues. Yet, depending on the primary target of such assessments, it might overlook the complexity and interdependencies of corruption problems and hence suggest ineffective solutions. In many cases, the availability of resources is a key factor in determining the scope of assessment. Choosing a methodology that considers the resources available, while also offering sufficient coverage of different corruption types, is one of the main challenges. Many state-of-the-art, Big Data methodologies work with massive amounts of data and require high-level analytical skills that may be out of reach to many anti-corruption agencies due to scarce resources and the need for affordable methodologies. Another challenge in conducting CRAs is the potential for biases and subjectivity in the selection of methodology and analytical focus. Depending on the organisation conducting the assessment and its priorities, certain types of corruption may be prioritised over others, which can result in an incomplete or inaccurate assessment of corruption risks.

Recommendations

First, taking a decentralised approach to CRAs can resolve some of the limitations imposed by available resources. When assessed public bodies conduct the CRA themselves, they carry out the analysis and provide the data they have locally. This means the methodology can be implemented with the right level of resources and data access. However, decentralised CRAs may suffer from inconsistencies in the application of the analytical tools and the weak comparability of results across organisations. Second, narrow, focused methodologies may be suitable when a CRA is being established in a country, as it is more feasible with limited resources and delivers focused results that can build trust among stakeholders. Over time, the methodology and corresponding corruption types can expand, building on the success of earlier assessments, and making the methodology gradually more appropriate and comprehensive. Additionally, it is important to establish clear criteria for selecting the methodology and analytical focus, and to ensure that the assessment process is transparent and inclusive of diverse perspectives and expertise. Finally, another recommendation to consider is to prioritise stakeholder engagement and feedback throughout the CRA process. This can help ensure the methodology and analytical focus align with the needs and concerns of relevant stakeholders, and can also promote transparency and accountability in the assessment process. This can involve conducting consultations with civil society organisations and other relevant stakeholders to gather feedback on the proposed methodology and analytical focus, as well as soliciting ongoing feedback and input throughout the assessment process. Additionally, involving relevant external experts can help achieve a balance within available resources without the need to expand regular staff.

Challenge: Follow-up mechanisms

Conclusive evidence of the effectiveness of CRA methodologies is lacking. Several intertwined challenges imply that a CRA’s recommendations may be ignored or only partially followed up by assessed entities. First, to observe whether a CRA’s recommendations and results have had an effect on practice, implementation of the methodology and dissemination of the results should be continuous. Since the forms of corruption are constantly evolving, there is a constant pressure to adapt the CRA, which unfortunately makes consistent comparisons over time, hence assessment of implementation results, challenging. Second, the basic lack of systematic monitoring of CRA recommendations’ impact – or even the ambiguity of what follow-up actions should look like – make CRA effectiveness hard to establish. In many cases, CRAs merely rely on assessed organisations’ good will and self interest in implementing recommendations.

Recommendations

Follow-up mechanisms should be part of all CRAs, such as the mandatory/voluntary implementation of recommendations. Accountability frameworks can be used – or built – to facilitate the implementation of CRA recommendations. First, follow-up mechanisms such as a requirement to implement recommendations should ideally be stipulated by law. In many cases, however, responses to CRA recommendations and findings are voluntary; here, communications, framing, and good working relationships among organisations are crucial. It is important to choose the most appropriate methodology and to be flexible: while nudging may work well in some contexts, stronger action may be required in others. Similarly, the need for thorough mitigation after risk assessment depends on the context and whether general recommendations are of interest to the agency itself. Second, the features of the broader accountability framework are decisive for the effectiveness of the CRA. Where a system of checks and balances, separation of powers, and civil service meritocracy are well-established, CRA recommendations are more likely to be effective. Furthermore, relying on the public to support the implementation of recommendations may be effective – even though interviewees had no consensus over this approach. For some anti-corruption bodies, it is particularly important that there is no public pressure on their assessments of other official bodies; this improves their willingness to cooperate and to be transparent with the assessing agency. At the same time, other agencies argue that to be truly effective, prevention of corruption can only happen through openness and accountability to the general public. As a middle ground, some recommend publishing only aggregated information or outstanding cases of non-compliance with anti-corruption regulations, while restricting organisation-specific information to internal reports.