This section describes the design of the study, data collection and data analysis processes for the survey. The design of the study envisioned two phases:
(i) identification of eligibility criteria;
(ii) development of a questionnaire.
These processes were inspired by the successful experience of the Ministry of Health of the Republic of Moldova in previous years. Data collection was conducted through an online interface and strengthened through face-to-face validation interviews. Data analysis was conducted by the research team in close collaboration with the IT specialist.
The research team comprised mainly Ministry of Health – Health Policy Analysis staff. Inputs such as the description and structure of the mid-term budgetary framework (MTBF) categories, as well as revisions to the general analysis, were provided by other relevant departments of the Ministry of Health.

1. Study design

1.1. Eligibility criteria

The Ministry of Health of the Kyrgyz Republic and the WHO Country Office conducted this research based on inputs collected from the development partners disbursing ODA.
Under the definition in the Organisation for Economic Co-operation and Development (OECD) Statistical Directives, para. 35, ODA includes all official transactions that:
1. are administered with the promotion of economic development and welfare of
• developing countries as its main objective; and
• are concessional in character;
2. and convey a grant element of at least 25%.

The report does not include any humanitarian or philanthropic assistance or sponsorship implemented in the health sector.
Development partners that met the criteria but did not disburse funds to the Kyrgyz health sector in 2015 are listed in Annex 1 but not included in this survey. To avoid double counting, when one development partner disbursed ODA funds on behalf of another, the development partner that carried out the final disbursement to the country is the one that reported for that project.

1.2 Questionnaire development and pilot

The research team developed a questionnaire to collect information on each development partner committed to the Kyrgyz health sector that had disbursed funds in 2015. During the pilot phase a draft version of the questionnaire was sent to several technical representatives of the development partner community providing ODA to the Kyrgyz health sector. The pilot phase and further consultations enabled the final version of the questionnaire to be refined and finalized at the end of June 2016.
An online version of the questionnaire was made available to all eligible development partners, with secure access through individual logins and passwords. The development partners’ questionnaire is available in Annex 2. A glossary of all the terms used in the questionnaire was also provided (see Annex 3). The completed versions of the questionnaire are available in Annex 4.

1.3 Questionnaire structure

Each section of the questionnaire was developed in cooperation with all the relevant departments and units of the Ministry of Health and in consultation with the WHO Country Office and independent consultants in order to accommodate multiple needs. As a result of these multiple inputs the questionnaire comprises ten sections.
Section I requests general information about the development partner agencies: their goals and key achievements; and the total amount of ODA they disbursed to the Kyrgyz health sector in 2015 classified by four different aid modalities – programme/project aid, sector budget support (SBS), pooled funds under SWAp, and other pooled funding.
Sections II, III, IV and V collate information about every programme and/or project run by each different development partner: date of start and completion; programme/project manager; mode of project implementation (through development partner office, public sector or other mode); and status of implementation as of 31 December 2015.
In addition, the questionnaire required the description of project/programme goals; current progress (target value – actual value); type of financing; and total budgets.
Financial efforts were quantified according to type of funding (i.e. technical assistance, investments and administrative costs) and further classified by delivering facilities, disease areas and risk factors. Appropriate filters were introduced in order to avoid mistakes and double counting. Section IV also asked development partners to reframe the project according to health-sector priority areas: health service delivery, resource generation, health financing, leadership and governance.
Information was also requested on the geographical coverage of each project/programme.
Sections VI, VII, and VIII focused on assessing alignment of aid with national policies and strategies; distribution of support by MTBF and other financial tools; use of public financial systems and procurement systems; mid-term (2016–2020) aid predictability within the MTBF subprogrammes; and development partners’ multiyear plans. These sections are intended to help the Ministry of Health to assess development partners’ alignment and harmonization.
The final sections of the questionnaire assessed donor coordination for joint missions and analytical work (Section IX); and development partners’ opinions and levels of satisfaction relating to coordination mechanisms and policy dialogue in the Kyrgyz health sector (Section X).
At the end of the questionnaire, participants were asked to provide their feedback on its structure and the appropriateness of the questions. This feedback will contribute to further development of the data collection system and improvement of future reports.

2 Data collection

2.1 Online data entry model

Data were entered through a web-based platform. The platform was adopted from the similar survey in the Moldovan health sector in 2011–2013, taking account of all the aspects and features of the health system of the Kyrgyz Republic.
The database was placed on the server of the Ministry of Health of the Kyrgyz Republic. Development partners were given a three-week timeframe for data entry starting from the beginning of September 2016. The online platform had advantages for both the development partners interviewed and the research team. Development partners could access the online questionnaire to enter and upload data at convenient times and resume the task without losing previous inputs. Development partners also had access to automatically generated PDF files – summary texts intended to enable easy visualization of the information provided and facilitate the data validation process. The system also avoided the difficulties associated with tracking reviews and comments that arise when different people work simultaneously on a questionnaire. The research team was able to monitor progress on data entry and (where necessary) send timely reminders; validate data more easily and quickly; and generate text files and update the database automatically.

2.2 Interviews

All development partners that met the criteria (see 3.1.1) were invited for interviews. These were conducted by the research team after the completion of data collection through the web-based platform at the end of September 2016 – either face to face or via e-mail. Interviews were held only after development partners had accessed the online questionnaire. The validation interviews had five aims:
1. to present the goal of the study, questionnaire and glossary;
2. to collect general comments and reactions relating to the overall process undertaken, and the difficulties encountered;
3. to go through all sections of the questionnaire and the respective definitions provided in order to achieve a good standard of data homogeneity;
4. to note relevant details that did not fit into the existing questionnaire in order to record where and how the design might be improved for future surveys;
5. to obtain extra information on specific issues that could not be recorded/standardized within the questionnaire, owing to their lack of homogeneity across development partners.

2.3 Data analysis

Data analysis comprised several methods, focusing on: (i) generating aggregate analysis for all development partners, their projects and their financial disbursements; (ii) providing qualitative analysis of development partners’ feedback on coordination processes; and (iii) listing key information for each development partner.
To standardize the financial information provided, development partners were asked to enter data in the original currency used for disbursements. When the database was generated the software automatically converted all currencies to United States dollars (US$), the reporting currency that the Paris Declaration used for all aid harmonization exercises. The software used the annual average exchange rate reported by the National Bank of the Kyrgyz Republic for 2015.
The IT specialist added two additional modules – one to generate individual questionnaires (text files) for each donor covered by the survey, the other to generate a database for the numeric variables inserted. The numeric variables were generated and analysed using Microsoft Excel®. Frequencies and cross-tabulations were used for data analysis and presentation.

2.4 Data quality

Data quality was ensured by several methods. During the design stage, the questionnaire passed through five rounds of reviews by the extended research team, along with a pilot process. During the data collection phase, the online web platform included several internal control mechanisms that prompted users to avoid common data entry mistakes. In addition, provision of a link to a glossary aimed to standardize interpretation of definitions and questions (see Annex 3). The data presented are those provided officially by the organizations covered by this report. Development partners also underwent a validation process, during which all the data were reviewed in order to avoid discrepancies caused by misinterpretations of the questionnaire or the glossary. Misinterpretations were a possibility because the questions have been formulated to accommodate two distinct needs – accurate enough to avoid misconceptions while allowing all development partners (with different vocabularies, reporting and accounting methods) to match the questions to their own purposes and to feel comfortable providing official answers. On completion of the validation processes, the relevant representatives had approved all the changes to the first version of the questionnaire submitted by individual development partners.
The joint effort of development partners and the research team during the validation processes enabled most development partners to complete all parts of the questionnaire. This guaranteed further homogeneity of the results. During the data analysis process all questionnaires were subject to a third level of data quality checking, using both exploratory analysis and further data cleaning to remove inconsistencies.