REQUEST FOR QUOTATION

RFQ Nº UNFPA/UKR/RFQ/18/12

 

Dear Sir/Madam,

 

UNFPA hereby solicits a quotation for the following service: EVALUATION OF UNFPA PSYCHOSOCIAL SUPPORT (PSS) TO SURVIVORS OF GENDER-BASED VIOLENCE IN UKRAINE THROUGH PSS MOBILE TEAM MODEL.

 

UNFPA requires evaluation to provide an independent and impartial assessment, complete with conclusions and recommendations, of the UNFPA-supported model of PSS mobile teams services and performance, as well as achievements, based on the conscientious, explicit and judicious use of the best available, objective, reliable and valid data and by accurate quantitative and qualitative analysis of evidence.

 

This Request for Quotation is open to all legally-constituted companies that can provide the requested services and have the legal capacity to perform in Ukraine, or through an authorized representative.

 

1. About UNFPA

UNFPA, the United Nations Population Fund (UNFPA), is an international development agency that works to deliver a world where every pregnancy is wanted, every childbirth is safe and every young person’s potential is fulfilled.  

 

UNFPA is the lead UN agency that expands the possibilities for women and young people to lead healthy sexual and reproductive lives. To read more about UNFPA, please go to: UNFPA about us

 

Terms of Reference (ToR)

Background and programme description

 

UNFPA, the United Nations Population Fund, is the United Nations sexual and reproductive health agency. Our mission is to deliver a world where every pregnancy is wanted, every childbirth is safe and every young person's potential is fulfilled. UNFPA strives to prevent and respond to gender-based violence through its work with policymakers, justice systems, health systems and humanitarian partners. UNFPA has been working for Ukraine since 1997, having delivered technical assistance for programmes in sexual and reproductive health, gender equality and gender-based violence (GBV), youth empowerment, population research, and humanitarian response.

 

Since November 2015, UNFPA and its national partners have been developing a system of psychosocial support (PSS) services for survivors of gender-based violence (GBV) in the government-controlled areas of eastern Ukraine, responding to the needs of women and adolescent girls affected by the armed conflict, including internally displaced persons (IDPs). This intervention has been part of the joint United Nations humanitarian response in Ukraine, and now is part of the broader UNFPA programme for developing and strengthening the  GBV prevention and response system at the national and regional levels in order to ensure that the majority of GBV survivors seek and have access to good quality survivor-centred services, while promoting a zero-tolerance approach to GBV in the society. The programme is accelerating the process of scaling-up and expanding mechanisms and frameworks that have been set up earlier in 2015-2017 through the UNFPA-led GBV response and prevention interventions.

 

In the framework of the programme, UNFPA has suggested a model of delivering PSS services on-site by the specially trained mobile teams (MTs) comprising of up to 3 members: psychologists and social workers. Provided with vehicles, the MTs can reach remote and underserved areas where they deliver psychological counselling, social support, case management information and awareness services, advisory and referrals to women in need to other service providers. MT members provide services to GBV survivors at the municipal centres of social services for family, children and youth, and do outreach information and awareness-raising work in local communities. As of September 2018, the model of mobile PSS services system is well-established and fully functional and has responded to over 40 thousand GBV cases with help and support. It is envisaged that UNFPA will shortly transfer the operational support of this model of mobile PSS services to national ownership and management, and they will be further developed and operated by the governmental counterparts.

 

In line with the corporate evaluation policy and wishing to make the exit as smooth as possible, UNFPA and its partners wish to undertake an external independent evaluation of the model of mobile PSS service provision (“PSS mobile teams”) supported by UNFPA, in order to analyse and verify its  functionality and impact, measure the achievements and flaws, draw out lessons learned, and most importantly to develop evidence-based practical recommendations for further institutionalization and development of the model integrating it into the existing system of PSS services provision in Ukraine. The evaluation will focus on the actual results achieved and will assess the implementation of the PSS services using such criteria as relevance, the validity of design, effectiveness, efficiency, sustainability, impact, factors affecting performance, alternative strategies and unanticipated results, and leadership and management (human, financial, systems). The evaluation is to be conducted by the competitively selected independent evaluator(s) unaffiliated with UNFPA or its partner organizations involved in the implementation of the PSS services system in Ukraine.

 

 

Situation Analysis

The armed conflict unleashed in eastern Ukraine in April 2014 has resulted in over 10,000 human casualties, massive violations of human rights, grave suffering and significant civilian displacement registered at some 1.3 million people. The conflict is yet to be resolved and continues to have a disproportionate impact on civilians living in the affected areas. Humanitarian organizations estimated that at least 3.7 million people have been affected by the conflict both directly and indirectly. Furthermore, the conflict has exacerbated the pre-existing systemic inequalities and poor performance of the country’s infrastructure and services.

 

People living along the “contact line”, those in the non-government-controlled areas (NGCAs), as well as internally displaced persons (IDPs) are the most affected population groups whose rights are threatened in many ways. Women and adolescent girls are more prone to GBV, exploitation and contracting sexually transmitted infections, including HIV infection. A study of GBV in five conflict-affected provinces of Ukraine organized by UNFPA in 2015 has revealed that internally displaced women had experienced GBV three times more frequently as local women who had not moved. Among the most prevalent forms of abuse during the conflict, the study respondents reported instances of humiliation, insults, intimidation, blackmail, verbal threats, physical violence (being hit or slapped), confiscation of money or property, confiscation of official documents, forced labour without pay or for a pittance, and being subjected to improper sexual comments. The study has identified and documented several cases of sexual violence. The UNFPA study identified members of armed groups as those committing over 50% of reported GBV cases. Focus group discussions also confirmed popular fears of men in uniform as potential perpetrators. Military personnel are frequently deployed in civilian settlements where they closely interact with host communities, which may result in violent behaviour, demand for sex work etc.). Their presence must, therefore, be considered an additional risk factor. The checkpoints at the demarcation line were regarded among the most dangerous locations in terms of vulnerability to all forms of violence.

 

The experience of violence can not only damage physical health. It could also result in grave psychological consequences, including life-long effects for the victim. Because of displacement, deprivation from normal life, these psychological effects are much more intense among IDPs. The most prevalent disorders identified in the study included intrusive memories (flashbacks), significant changes in sleep patterns and repeated nightmares, and a permanent feeling of fear or guilt. These problems could adversely influence resilience, potentially provoking the risk of increased domestic violence and violence outside the family. Therefore, the need for accessible PSS services is urgent not only for the survivors of violence but for all people living in the conflict-affected areas.

 

There is no common practice to seek assistance from specialized institutions in cases of violence. The overwhelming numbers of GBV survivors surveyed in the UNFPA study have not applied for psychological or legal support, neither have they applied for medical assistance. The main causes for not applying for assistance according to respondents were the unawareness of the availability of services, absence of proper institutions in communities and mistrust in service personnel. One in ten IDP survivors hadn’t sought the assistance of law enforcement bodies because of the fear of further violence. Most frequently, GBV survivors share their experience with their closest people only (such as family members and friends). The emotional support provided by friends and loved ones is regarded as the most effective coping strategy in cases of violence. Some expectations of help were expressed for the assistance of psychologists and representatives of NGOs and women’s support groups. The study highlighted a widespread stigma associated with rape/sexual assault and low public trust towards state-run service providers. According to the study, health care and psychological and social services are mostly concentrated in the provincial capital cities, while remote communities have no affordable solutions when they need to get such services. The United Nations Humanitarian Needs Overview for Ukraine document of 2015 has identified access to friendly care services for GBV survivors, including post-rape care, access to essential health care for pregnant women, and access to HIV/STI prevention and treatment services as the most urgent health-related lifesaving humanitarian needs.

 

UNFPA Response (Subject to be Evaluated)

Based on these considerations and the UNFPA global mandate to lead the GBV response including in humanitarian settings, UNFPA has developed a complex programme to strengthen the capacity and expand coverage of service providers to comprehensively respond to GBV cases and prevent future acts of violence using a survivor-centered approach, ensuring free and inclusive access for all GBV survivors to good quality essential services (psychosocial support, health, legal aid etc.). One of the focus areas of the programme was scaling up free-of-charge, safe and confidential psychosocial support to GBV survivors while reaching those left furthest behind. The UNFPA led GBV response services (such as “PSS mobile teams”, “safe space solutions/shelters”, health service delivery points, telephone hotline) launched in humanitarian phase as humanitarian short-term solutions, gradually became a model for national level expansion and replication. Since 2017, UNFPA, jointly with government counterparts started working on expanding, developing existent ones and institutionalizing such models. The recently adopted Law of Ukraine on Preventing and Combating Domestic Violence (#2229-VIII) opens the possibility for the institutional recognition of such specialized services and envisages improvement and further development of services for emergency and long-term protection of GBV and domestic violence (DV) victims, including through establishment of such response services as PSS mobile teams.

 

The PSS purpose is to help violence survivors overcome psychological distress, develop their coping mechanisms, protect them from discrimination, abuse, exploitation and violence in the future, provide them with information and pathways to manage their cases and receive all necessary care and support in an integrated manner. This approach makes women and adolescent girls more resilient, self-confident and capable to confront GBV. The programme also addressed GBV response coordination and integration, referral system development, establishing physical protection and shelter facilities, medical care including HIV/STI treatment (e.g. post-exposure prophylaxis) and legal counselling. The programme is funded by a donor coalition that includes UNFPA, UNICEF, governments of the United Kingdom, Canada and Estonia.

 

The mobile teams provide both urgent and regular social and psychological care services to all GBV survivors but especially to those who suffered from the armed conflict in eastern Ukraine. Services are provided in a safe environment and are guided by basic principles of care provision. Assistance is rendered through professional counselling when visiting clients at their homes or other places of residence, as well as at dedicated and adapted premises in centres of social services for family, children and youth, or other facility as may be decided by the community. If required, MTs refer clients to relevant psychological, legal or health care service providers according to the clients’ needs and can engage police officers to respond to GBV cases. PSS services are provided free of charge, according to national legislation. The MTs are equipped with vehicles to make field visits and facilitate outreach work.

 

 

The PSS MTs are operated by the Ukrainian Foundation for Public Health, the UNFPA implementing partner responsible for this component of the GBV response programme. Regulatory support and oversight of the service are provided by the Ministry of Social Policy of Ukraine and the UNFPA GBV programme is being coordinated with the National Police / Ministry of Interior of Ukraine as related to law enforcement actions. The MTs are operating on-call and make regular planned outreach visits within their geographic area of responsibility for information, education and communication activities. The implementation of the PSS service delivery system has been coordinated within the United Nations humanitarian response architecture through the UNFPA-led GBV sub-cluster, part of the protection cluster, and in the wider development context through the Gender Theme Group that brings together governmental agencies, non-governmental actors, academic and research institutions, development and donor communities working in Ukraine to promote gender equality.

 

The PSS service is entrusted with the following professional tasks that are performed in close collaboration between the MTs and local centres of social services for family, children and youth:

 

• Provision of PSS to GBV survivors according to their individual needs, on-site or at a service facility
• Urgent response and emergency PSS support in acute GBV cases, critical case management and transportation of survivors to safe space solutions
• Provision of information to clients on the available emergency and regular services (medical, social, psychological, legal etc.) they can receive to cope with the consequence of the GBV case, and on respective service providers
• Awareness raising and education on GBV, its consequences and means of prevention
• Cooperation with relevant authorities, services, institutions and organizations for effective referrals of GBV survivors for timely and effective assistance

 

Besides GBV survivors, MTs work with their family members, including children. More specifically, MTs provide the following primary level services:

 

• Informing about nearest medical care facilities where post-exposure prophylaxis (PEP) kits are available for the prevention of HIV/AIDS, sexually transmitted infections and unwanted pregnancy during the first 72 hours since unwanted and/or unprotected sexual intercourse
• Informing about GBV types and features
• Counselling on ways to prevent GBV and coping with its consequences
• Diagnostics of a socio-psychological status of GBV survivors and assessment of risks of repeated violence
• Counselling on social and legal aspects of GBV response, including through referrals to other specialists
• Providing psychological aid, psychological counselling, including emergency assistance
• Providing social support, including through referrals to other service providers
• Responding to GBV against children, providing first psychological aid, registering cases with relevant authorities, providing required information
• Developing safety plans for GBV survivors who are at risk of repeated violence
• Facilitating additional aid and services for GBV survivors (medical, information, legal, financial etc.)
• Conduct community outreach through regular visits, including house-to-house to disseminate information on available services for women and girls, planned activities (lectures in educational facilities, etc), invite participation
• Conducting public information, education and communication events related to human rights, gender equality, GBV prevention and response for GBV survivors, local communities, school and university students

 

All MT members received special training on GBV, survivor-centred psychosocial support provision, crisis management, national GBV response mechanism design and use of the GBV response referral system. The MTs also received equipment necessary for performing their duties, including uniforms and vehicles. UNFPA covers the PSS MTs maintenance costs.

 

Using mobile terminals (tablet PCs with specialized software), the MTs use a common GBV information management system (GBV IMS) where confidential records of each case are collected through dedicated intake forms and stored in a database. The system allows for a more in-depth analysis of MTs performance and features of GBV. Based on the data collected by the MTs, UNFPA produces monthly progress reports on the PSS service performance (available through the UNFPA Country Office in Ukraine).

 

In line with the United Nations Humanitarian Response Plan, initially the PSS services have been established in 5 provinces of Ukraine geographically closest to the armed conflict zone and having received the lion’s share of IDPs:

 

• Donetska (MTs in Mariupol, Bakhmut, Kostiantynivka, Kramatorks, Novogrodivka, Selydove, Slovyansk and Volnovakha)
• Luganska (MTs in Severodonetsk, Kreminna, Lysychansk, Novoaidar, Popasna, Rubizhne, Stanytsia Luganska and Starobilsk)
• Kharkivska (two MTs in Kharkiv, MTs in Izium and Lozova)
• Dnipropetrovska (MTs in Dnipro, Kamyanske, two MTs in Kryvyy Rig, MTs in Nikopol and Pavlograd)
• Zaporizka (MTs in Zaporizhzhia, Berdiansk and Melitopol)

 

As the situation evolved, the PSS services have been expanded into other areas of Ukraine. In 2017 the PSS service covered the following provinces:

 

• Kyivska (MTs in Bila Tserkva, Borodianka, Vyshneve)
• Odeska (two MTs in Odesa)
• Khersonska (MTs in Kherson, Chaplynka, Nova Kakhovka and Genichesk),
• Vinnytska (MTs in Vinnytsia, Bar, Gaysyn and Mogyliv-Podilskiy)

 

In 2018 three more provinces joined the programme:

 

• Lvivska (MTs based in Lviv, Sokal, Zhydachiv and Zolochiv) and
• Mykolayivska (two MTs in Mykolayiv and one in Pershotravensk)
• Rivnenska (MT in Dubno)

 

Along with that, two new MTs have been established in 2018 in Odeska province – in Berezivka and Izmail. The total number of MTs has grown from 21 covering 5 provinces in 2015 to 50 covering 12 provinces in 2018. Contacts and coverage information of the MTs are available at the Rozirvy Kolo project web page (being updated as more teams join).

 

As of September 2018, the PSS MTs provided response to 40,388 GBV cases among women (89%) and men (11%) who are both local non-displaced residents (70%) and IDPs (29%), managing cases of psychological (48%), physical (34%), economic (17%) and sexual (1.8%) violence. It is worth mentioning that as many as 64% of PSS clients had not previously reported their GBV case anywhere else but to the MTs, therefore the UNFPA PSS programme has been for them the first available and sole solution to access the help they needed. Nowadays, countrywide the PSS MTs respond to about 2,000 GBV cases each month.

 

 

The work of the UNFPA PSS service received positive media feedback across Ukraine. The following media materials tell their stories about the PSS MTs:

 

• UNFPA news
• Kramatorsk news
• Lviv news
• Bila Tserkva news
• Mariupol news
• Zolochiv news

 

Several videos produced by UNFPA feature the PSS MTs’ work in Donetsk province and were aired on local TV stations as public service announcements in summer 2018.

 

Having built and supported a fully functional model, in the coming months UNFPA plans to transfer the ownership and operational support of the mobile PSS service to its national counterparts, the Ministry of Social Policy of Ukraine, local governments in respective provinces and districts and respective communities (hromadas). To facilitate this process and ensure the sustainability of the system, an independent forward-looking evaluation of the mobile PSS service model performance is required that would critically review in details the design and implementation of the PSS service, draw conclusions and lessons learned and then elaborate feasible options and recommendations for its further institutionalization and development based on the available evidence.

 

 

The purpose of the planned evaluation exercise is to independently and transparently identify appropriate information sources and generate substantive reliable evidence on various performance aspects of the PSS delivery model through MTs established by UNFPA. The evaluation must seek to explore, describe and possibly measure the change that the PSS MTs services have driven in the lives of GBV survivors and other populations served, the contribution made to the psychological and social well-being of their clients, the difference they made in the national GBV response machinery. The exercise will strive to objectively assess the PSS programme’s achievements and evaluate the programme using the specific criteria as set forth below. UNFPA expects the evaluation to highlight the mobile PSS good practices and lessons learned, comprehensively analyse the design, implementation and sustainability of the mobile PSS services system, based on evaluation findings draw evidence-based conclusions and recommendations as to possible options of further institutionalization and development of the system. Along with UNFPA, programme donors and implementing partners, the evaluation findings will be presented to programme beneficiaries such as the Ministry of Social Policy of Ukraine and Ministry of Interior of Ukraine in the context of strengthening the national GBV response mechanisms.

 

 

The objective of this evaluation is to provide an independent and impartial assessment, complete with conclusions and recommendations, of the UNFPA-supported model of PSS mobile teams services and performance, as well as achievements, based on the conscientious, explicit and judicious use of the best available, objective, reliable and valid data and by accurate quantitative and qualitative analysis of evidence.

 

Primarily, the evaluation will explore and determine the relevance, effectiveness, efficiency, impact and sustainability of the model of PSS MT services, always having in mind the change they have been expected to bring to people’s lives and the added value of work organized and supported by UNFPA. Additionally, the PSS programme’s design, coverage, coordination, service quality and implementation modality and contribution to the work of the protection cluster of Ukraine’s humanitarian response need to be reviewed and assessed.

 

The evaluation will be guided by the United Nations Evaluation Group Norms and Standards for Evaluation and will apply the internationally acknowledged OECD-DAC Criteria for Evaluating Development Assistance. When planning and implementing the evaluation, the Inter-Agency Guide to the Evaluation of Psychosocial Programming in Humanitarian Crises should be extensively consulted by the evaluators, especially with regard to selecting and using specific questions and indicators.

 

Below is the list of principal evaluation criteria and corresponding key evaluation questions to find answers to. These are rather typical questions that may later be revised, refined and/or adjusted in consultations between the evaluators, UNFPA and other PSS programme stakeholders.

 

Criterion	Evaluation Questions
Relevance	How relevant are the proposed services in the local context from the point of view of recipients? How well do they address the needs of the clients? How were the potential beneficiaries involved in the programme design? How appropriate is the proposed approach and model to service delivery in the current operating environment? How well is the programme aligned with the national priorities in combating GBV and domestic violence and reflects the national legislation? How socially and culturally acceptable are the PSS mobile teams’ services? Were the services gender- and age-specific? What are the beneficiaries’ and stakeholders’ perceptions of the UNFPA programme and services provided?
Effectiveness	How well has the programme addressed the priority problems identified at the planning stage? Has the programme achieved its planned objectives in time? To what extent have the objectives been achieved? Were the achievements measurable through established indicators? What new capacities in people, communities and services have been developed by the PSS programme? What were the major factors influencing the achievement or non-achievement of the objectives? What are the intended and unintended effects of the programme? What is the quality of services provided as perceived by the beneficiaries?
Efficiency	Did the actual or expected outputs justify the costs incurred? How cost-efficient were the programme activities? Did the programme demonstrate the most economical use of resources? Have all programme resources (financial, human, technical) been utilized in full as planned? What is the total cost of PSS MT response per visit (per case)? Is there any feasible possibility to decrease the cost? Was the programme implemented in the most cost-efficient way compared with similar interventions? Could there be an alternative, more cost-efficient, programme approach for achieving the same results?
Impact	What has changed in the lives of people served by the programme and the general population because of the programme implementation? Has the wellbeing, coping capacity and resilience of the programme’s primary beneficiaries improved? What new skills and knowledge generated by the programme are most valued by the beneficiaries? Has the national GBV response mechanism been improved because of the PSS programme implementation? What was the programme’s impact on the communities affected?
Sustainability	How was the mobile PSS services sustainability strategy considered in the programme design? Was the programme based on the development and use of local capacities and mechanisms? Is there a programme exit strategy in place? Is it likely that programme achievements will be sustained after the withdrawal of external support? Are involved counterparts willing and able to continue programme activities on their own? How feasible would the mobile PSS services operation be after the programme completion? What are the resource and institutional/regulatory requirements for ensuring the programme’s sustainability and full national ownership? What are the best ways to sustain the mobile PSS services after a withdrawal of external support?
Design	Does the programme reflect an explicit theory of change that offers appropriate solutions to the identified problems and needs? Is the programme’s chain of results logical? Do the programme expected results comply with the SMART criteria? Are resources (financial, human, technical) and strategies identified appropriate and adequate to achieve the results? Is there a robust programme monitoring system in place?
Coverage	How was the programme coverage planned? Has the coverage been adequate compared with the resources available? How many people and institutions have benefited from the programme? Could the programme have covered more areas and served more people? Can an estimate be made regarding people served vs people in need?
Coordination	How well was the programme coordinated with the work of other humanitarian and development actors? What were the coordination mechanisms? Are the programme activities complementary to other UN and donor interventions? Are there any signs of activity duplication or overlaps? How well is the programme integrated into the national GBV response mechanism?
Quality	To what extent do the PSS services comply with the applicable international standards, e.g. the Interagency Standing Committee Guidelines on Mental Health and Psychosocial Support in Emergency Settings and Guidelines for Integrating Gender-Based Violence Interventions in Humanitarian Action, as well as the applicable national standards, norms and regulations?
Implementation modality	How adequate was the selected PSS MT model implementation modality for achieving the expected results? How effective was the programme management as related to the model implementation? Was the programme adequately staffed? How satisfactory were the programme monitoring and reporting?
Contribution to humanitarian response	To what extent has the PSS programme contributed to Ukraine’s humanitarian response plan, the work of the protection cluster? How has the programme contributed to protecting women and adolescent girls from GBV and domestic violence? How does the government at the central and local level assess the humanitarian contribution of the programme?

 

 

The evaluation will cover the time period from November 2015 when the PSS MTs have been operationalized to present time, and geographically all districts of Ukraine where MTs have operated and for which management information is available at the time of evaluation, as well as Kyiv city where main offices are located. The process will engage all PSS programme stakeholders at the central and local levels, i.e. UNFPA, UNICEF, UN Women, UN OCHA, UNHCR, ICRC, MdM, IMC, People in Need, Ministry of Social Policy, Ministry of Interior, National Police, local state administrations, local service providers, GBV response referral networks, PSS MT members, PSS service clients, local communities where MTs operate, possibly local media. The evaluation will be focused on the work of the UNFPA-supported PSS MTs and all involved implementing partners and will cover other components of the UNFPA GBV response programme as and when necessary. While all stakeholders are important sources of information for the evaluation, the direct beneficiaries of the PSS programme will be put in the centre of attention.

 

 

The evaluation team is expected to develop a specific and detailed evaluation framework with methods and instruments to be discussed and agreed on by UNFPA and programme stakeholders. The methodology should engage a mix of data collection and analysis methods, with a preference for quantitative methods where appropriate and possible. The methodology should be chosen with a clear intent to provide credible answers to the evaluation questions. The methodology should ensure that the information collected is valid, reliable and sufficient to meet the evaluation objectives and that the analysis is logically coherent and complete (and not speculative or opinion-based).

 

The programme data collected should be verified through triangulation (use of multiple independent sources) wherever possible to ensure its objectivity. For assessing the PSS programme impact, comparison groups representing people as similar to the PSS MTs clients as possible but from areas not covered by the UNFPA or any other GBV PSS programmes will be identified and engaged in the data collection to allow for a decent comparative analysis.

 

The following methods of data collection are suggested (the list if for reference only; the evaluators may suggest their methods and tools as appropriate in full compliance with the requirements of these terms of reference):

 

• Desk review of programme documentation, available reports and thematic publications (e.g. programme documents, work plans, budgets, programme and event reports, programme data sheets, GBV information management system data, programme review reports, donor reports, humanitarian response documents and publications, official statistics, thematic research papers etc.)
• Key informant interviews, including representatives of the central and local governments, UNFPA and programme management
• Structured and semi-structured interviews with services providers representing the GBV referral networks (e.g. PSS MT members, police officers, legal advisors, doctors, social workers who have cooperated with the PSS MTs etc.)
• Interviews with individual PSS clients and comparison groups
• Focus group discussions with PSS clients and comparison groups
• Physical observations
• Case studies of most exemplary cases identified

 

A sample outline of the evaluation framework is provided below for guidance.

 

Evaluation Question	Sub-Question	Indicator /Criterion	Data Source	Data Collection Method	Sampling Plan

 

The suggested methodology must indicate, in analysing data, what benchmarks will be used in making the assessment for each evaluation criteria or question. The methodology should also clearly specify the roles that each of the stakeholders will play during the evaluation exercise. Given the sensitivities of the violence-related issues, the evaluators will be requested to take into account ethical considerations when collecting information and explain how this will be done.

 

Upon agreement with UNFPA and programme stakeholders, the evaluation framework will translate into an evaluation calendar workplan. UNFPA commits to provide the evaluators with all available UNFPA and implementing partner documents that may be relevant for the evaluation.

 

 

The PSS programme evaluation will be managed by a structure that includes the following:

 

• Evaluation Manager – the UNFPA Assistant Representative who organizes the evaluation on behalf of the UNFPA Country Office in Ukraine.
• Evaluation Reference Group – a group representing key PSS programme stakeholders who can provide different perspectives and knowledge on the programme design and performance and therefore enhance the relevance, quality and credibility of the evaluation processes. The group will include the Evaluation Manager, senior managers from the UNFPA implementing partner organizations engaged in the PSS programme, representatives of concerned central level government agencies (Ministry of Social Policy, Ministry of Interior, National Police), the Programme Manager responsible for the UNFPA GBV response programme, the GBV Sub-Cluster Coordinator/GBV Technical Advisor, and possibly a representative of a collaborating UN agency (e.g. UN Women or UNICEF). Decisions of the group will be made on a consensual basis with a simple majority quorum.
• Evaluation Team – experts contracted by UNFPA for conducting the evaluation in full compliance with the provisions of these terms of reference (ToR) being supervised by and reporting to the Evaluation Manager and guided by the Evaluation Reference Group. The Evaluation Team will develop the evaluation methodology and framework, conduct the evaluation and present findings and recommendations in a substantive evaluation report. The team will have an Evaluation Team Leader responsible for coordinating the teamwork, collecting and compiling inputs from team members, and for all official communications on behalf of the team.

 

The main oversight responsibilities of the Evaluation Manager and the Evaluation Reference Group are outlined presented below. Whenever possible, the Evaluation Reference Group will communicate virtually/electronically and will hold meetings as may be required.

 

Evaluation Manager

Evaluation Reference Group

Lead the development of the evaluation ToR Supervise and coordinate the selection and recruitment of the Evaluation Team safeguarding the independence and impartiality of the evaluation exercise Facilitate and convene Evaluation Reference Group meetings Manage and coordinate the evaluation process and events among all stakeholders Ensure that the evaluation and its products meet quality standards Coordinate and facilitate the work of the Evaluation team with all stakeholders and other relevant counterparts, including UNFPA units/staff Facilitate access of the Evaluation Team to all available information and documents related to the evaluation subject, as well as to implementing partners and key informants Review and approve the evaluation deliverables Disseminate the evaluation findings and take necessary action on the recommendations through a management response

Review and approve the evaluation ToR Brief the Evaluation Team before the commencement of the evaluation Support the Evaluation Manager and Evaluation Team in performing their duties, flagging issues and suggesting solutions Review and approve the inception report Review and comment on preliminary findings of the evaluation, as well as on draft evaluation reports Review and comment on draft recommendations of the evaluation to enhance their feasibility, acceptability and ownership Disseminate and follow up on evaluation finding and recommendations as appropriate

 

The language of official communications with UNFPA on all evaluation-related matters will be English. Some limited logistical support to the Evaluation Team may be provided by the UNFPA GBV response programme personnel subject to the availability of resources.

 

 

All Evaluation Team members should respect and comply with the ethical principles for evaluation, which includes the obligations to behave ethically in terms of:

 

• Intentionality: bearing in mind the purpose, usefulness and necessity of the evaluation at all its stages
• Avoiding conflict of interest: upholding the principles of independence, impartiality, credibility, honesty, integrity and accountability
• Interactions with participants: engaging appropriately and respectfully with participants in all evaluation processes, upholding the principles of confidentiality and anonymity and their limitations, dignity and diversity, human rights, gender equality, avoidance of harm, especially with regard to sensitivities connected with GBV and domestic violence
• Evaluation processes and products: ensuring accuracy, completeness and reliability, inclusion and non-discrimination, transparency, fair and balanced reporting that acknowledges different perspectives
• Discovery of wrongdoing: discreetly reporting the discovery of any apparent misconduct to UNFPA

 

 

UNFPA expects the Evaluation Team to provide information, produce and submit the evaluation reports, presentation of findings and recommendations and infographics as explained below in this section. These documents should be submitted to the Evaluation Manager in soft copies in English before the deadlines agreed with UNFPA upon the approval of the inception report. Text documents should be prepared preferably in Microsoft Word on A4 size pages will all page margins of 2 cm using the Calibri or Times New Roman font of size 11 with single line spacing and paragraphs separated with a blank line. Layout design of the deliverables is not required but most welcome.

 

1. Inception Report

 

The inception report should provide the evaluation framework and describe in details the proposed evaluation design, methodology, data sources and collection methods, sampling, general evaluation implementation plan with a timeline, field visits plan, meetings plan and list of key informants, deadlines for submitting the deliverables, and the suggested structure/contents of the evaluation report. The inception report should also include the initial findings resulting from discussions with the Evaluation Reference Group, UNFPA personnel and desk review of the PSS programme documentation. The report will be used to establish mutual understanding and agreement between the evaluators, the Evaluation Manager and the Evaluation Reference Group. 

 

The draft inception report must be submitted by the Evaluation Team within 8 days after having the initial briefing from the Evaluation Reference Group. It will then be reviewed and commented on by the Evaluation Reference Group and returned to the Evaluation Team within 3 days. The final inception report will constructively address all the feedback received and be submitted to the Evaluation Manager within 3 days after receiving the feedback on the draft. Therefore, work on the inception report should be completed in 14 days after the initial briefing.

 

2. Field Work Debriefing

The Evaluation Team will conduct a debriefing meeting for the Evaluation Reference Group immediately after the completion of the evaluation data collection activities (field work). The debriefing date should be agreed with the Evaluation Manager at least 7 days in advance. The debriefing should provide detailed information on the activities completed, contacts made, data collected, facilitating and constraining factors that have influenced the field works, and preliminary findings and recommendations. The Evaluation Reference Group’s reflections on the information provided during the debriefing will be useful for analysing the data, drawing conclusions and elaborating recommendations.

 

3. Evaluation Report

The evaluation report is expected not to exceed 50 pages excluding annexes and should follow the structure approved by the Evaluation Reference Group, unless otherwise agreed in writing. The report must contain an executive summary (suggested maximum volume 3 pages) that briefly presents the PSS programme and its context, evaluation purpose and objectives, its methodology and key findings, conclusions and recommendations. Preferably, the executive summary should be written in short sentences, in a clear and simple language and thus be media-friendly. The report must only contain information relevant for the evaluation purpose and avoid including information of no direct relevance to the analysis. Evaluation readers should be able to easily understand:

 

• What was evaluated and why (purpose and scope)
• How the evaluation was designed and conducted (evaluation questions, methodology and limitations)
• What was found and on what evidence base (findings and evidences)
• What was concluded from the findings in relation to main evaluation questions asked, and how such conclusions were drawn (good practices and conclusions)
• What and why was recommended (recommendations)
• What could be learned from the evaluation if any (lessons learned).

 

Special attention in the report should be paid to recommendations. They should be strictly based on evidence collected during the evaluation and analysis made, follow exclusively from the evaluation findings and conclusions and not be based merely on opinions. Recommendations could include strategic directions and operational solutions. They should be practical and actionable, and very clear about who should take the proposed action, albeit not too prescriptive. Therefore, draft recommendations should be discussed in detail with potential implementers to secure their acceptability and feasibility and foster understanding, ownership and commitment of those who will act.

 

The evaluation report should be prepared through a minimum of three iterations. The first draft evaluation report must be submitted for review, comments and suggestions within 14 days after the fieldwork debriefing. The Evaluation Reference Group commits to take up to 5 days for providing consolidated feedback on the report to the evaluators. Within 5 days after receiving this feedback, the evaluation team should incorporate it and submit the second draft evaluation report. The Evaluation Reference Group will then review the document and finalize it on a no objections basis, with possible involvement of the UNFPA Regional Monitoring and Evaluation Advisor. At this stage, further comments and suggestions will be addressed by the evaluators on an ad hoc basis. After these two rounds of reviews, the final evaluation report will be produced. Therefore, the entire report writing timeframe should not normally exceed 24 days.

 

 

 

4. Evaluation Presentation

A PowerPoint presentation based on the final evaluation report should summarise the entire evaluation process, purpose, objectives, methodology, data collection methods, findings, analytical conclusions and recommendations. The presentation must be prepared in both English and Ukrainian identical versions. The Evaluation Team Leader may be requested and should be ready to present in a public meeting (e.g. with the government, donors or UN agencies).

 

5. Information Graphics

The evaluation team should prepare an infographics material summarising the quantitative and qualitative findings of the evaluation using text, charts and images/icons. The infographics should be simple and media-friendly and avoid using technical terms. The material must be prepared in English and Ukrainian identical versions in consultations with the Evaluation Manager and UNFPA communications and advocacy officer.

 

Below are the suggested key milestones for the evaluation process, which ideally should not extend beyond two months. Timing shown in the table is relative to the day the contract for evaluation services is signed by all parties. The Evaluation Team should propose a calendar plan of works and deadlines for review and approval by the Evaluation Reference Group as part of the inception report. Once approved, the evaluation calendar plan can be amended only if agreed in writing between UNFPA and the Evaluation Team provided that sound justifications are provided by the evaluators.

 

Milestone	Responsibility	Timing (Days)
Briefing for the Evaluation Team	ERG	3
Inception Report submission (draft)	ET	11
Feedback on draft Inception Report	ERG	14
Inception Report submission (final)	ET	17
Evaluation fieldwork (data collection) debriefing	ET	31
Evaluation report submission (first draft)	ET	45
Feedback on the first draft Evaluation Report	ERG	50
Evaluation report submission (second draft, presentation and infographics)	ET	55
Finalization of Evaluation Report	ERG/ET	60

 

 

All costs of the evaluation will be covered by UNFPA through a contract (or contracts) for evaluation services. Evaluators could be contracted either with individual consultancy contracts as physical persons or with a contract for professional services as a legal entity. The evaluators are supposed to use their own equipment and tools for this assignment (computers, printers, photocopiers, cameras, voice/video recorders etc.) and make their own travel arrangements for travels. UNFPA is not in a position to provide visa support or facilitate receiving work permits for evaluators.

 

Evaluation contract payments will be made by UNFPA via bank transfers through the United Nations Office in Ukraine based on review and approval of the deliverables by the Evaluation Manager in consultations with the Evaluation Reference Group. The contract amount will be paid in three instalments as follows:

1. 20% upon approval of the final Inception Report
2. 30% upon approval of the first draft Evaluation Report
3. 50% upon acceptance of the finalized Evaluation Report, presentation and infographics

 

 

In response to the UNFPA announcement (request for quotations), interested companies or groups of consultants should prepare and submit their applications in two separate documents comprising a) technical and b) financial proposals.

 

Technical proposals should meet all the requirements of these terms of reference ensuring that the purpose, objective, questions, scope, criteria, deliverables, management and financial arrangements of the evaluation are considered. A technical proposal should demonstrate the understanding of the assignment by the evaluators and explain the proposed approach to organizing and managing the works, evaluation methodology, data sources, sampling and data collection methods and tools, data analysis procedures and criteria for making judgments, as well as the proposed structure/contents of the evaluation report and how it will be composed. The technical proposal should also provide a work plan and timeline, the composition of the Evaluation Team with updated CVs of all members, and links or soft copies of two most recent evaluations performed by the proposed team. An e-mail with the technical proposal must be clearly marked “TECHNICAL PROPOSAL” and should NOT contain any financial information, otherwise, they will not be qualified for consideration.

Financial proposals should provide a detailed description of the proposed evaluation costs in Ukrainian Hryvnias with proposal validity term of 30 days, including:

• Hourly rates for each evaluator of the team
• Proposed number of hours for each evaluator on this assignment and specification of work to be performed
• Operational support costs (e.g. transportation, subsistence, premises rent, communications, interpretation/translation if any, expendables, stationery)
• Any other costs that need to be covered to make the evaluation exercise a success, with detailed justifications

Travel expenses should be based on the most direct and economical fares and should not exceed the applicable rates established by the United Nations for Ukraine. The financial proposal must be submitted together with the technical proposal in a separate document clearly marked “FINANCIAL PROPOSAL”.

 

Details instruction for submission of bids is specified in section IV.

 

The applications must be sent to the UNFPA Country Office in Ukraine to the e-mail address and by the deadline as indicated in the request for quotations announcement. Applications received after the deadline will be returned to applicants unopened.

 

An evaluation committee will be established by UNFPA to evaluate all received applications. Applications will be reviewed in a two-stage process: technical proposals will be evaluated and rated before opening of financial proposals. Only those proposals found technically compliant will be subject to comparative financial evaluation.

 

The applications will be evaluated with 60% weight assigned to technical proposal score and 40% to financial proposal score, and according to the following criteria and scores:

 

Technical proposals (100 points max):

Overall response (20 points max)

• General quality and completeness of the proposal vis-à-vis the terms of reference and request for quotations requirements
• Applicant’s understanding of the evaluation subject, purpose, objectives, scope, expected deliverables
• Applicant’s background, official registration, certifications, memberships etc.              

Proposed methodology and approach (50 points max)

• A proposed approach to organizing and managing the evaluation
• Proposed methodology, data sources, sampling and data collection methods and tools, data analysis procedures and criteria for making judgments
• Proposed structure/contents of the evaluation report and how it will be composed
• Proposed work plan and timeline
• Deliverables are addressed as per TOR; proposed timelines are met

 

Technical capacity of the Evaluation Team (30 points max)

• Range and depth of experience with similar evaluations
• Academic qualifications and job record
• Competencies and skills relevant for the evaluation (e.g. communications, analysis, data management, report writing, software use)
• Language proficiency

 

Only those technical proposals achieving the score of 60 points and above will be considered as qualifying for evaluation of the financial proposal.

 

Financial Proposals (100 points max):

Financial proposals should follow the results-based budgeting approach. They will be assessed based on their clarity, completeness, level of detail and appropriateness. The maximum number of points shall be scored to the lowest price proposal among all technically qualifying applications. Other financial proposals will receive scores according to the following formula:

 

Financial score =	Lowest price	x 100 (Maximum score)
	Price being scored

 

The total score of each application will represent the weighted sum of its technical and financial scores as follows:

Total Score = [60%] Technical score + [40%] Financial score

 

To systematically ensure that the results of the evaluation are used to inform programming and strategic and policy decisions, the UNFPA Country Office in Ukraine, collaboratively with the PSS programme stakeholders, will prepare and implement a management response to the evaluation findings. It may include a review of and modifications to the ongoing programmes and projects of UNFPA and other actors, consideration of findings in new programmes and projects, publication of good practices and lessons learned, issuance of advisory notes etc. The evaluation results will be used by the national governmental counterparts when developing a strategy for further strengthening of the national GBV response mechanisms and expansion of the PSS services in Ukraine.

 

The evaluation report will be disseminated by UNFPA to all PSS programme stakeholders, including the donor community, as well as to relevant media outlets, in full and as an executive summary, and will be made available for free public access at the UNFPA website.

 

 

The Evaluation Team will comprise experts, national or international, with a solid monitoring and evaluation background and respective practical experience of evaluating international humanitarian and development interventions or will represent a specialized agency. One of the team members will be assigned with the Team Leader responsibilities. The team should have the following qualifications and skills mix:

 

• Academic background or special training in monitoring and evaluation, preferably in the international humanitarian and development context
• Training on or practical involvement in GBV prevention and response interventions, preferably in humanitarian contexts
• Excellent knowledge of evaluation principles, norms, standards, methodologies, designs, ethics and practices
• Technical evaluation skills
• Evaluation management skills
• Proven experience of conducting programme or project evaluations, preferably in the international humanitarian and development context, ideally in gender equality or GBV response
• Excellent knowledge of Ukraine’s systems of social protection and social services
• Data management and analytical skills
• Communication and interpersonal skills
• Time management skills, ability to respect set deadlines
• Excellent writing skills
• Perfect knowledge of written and spoken English and Ukrainian

2. Questions

Questions or requests for further clarifications should be submitted in writing to the contact person below:

 

Name of contact person at UNFPA:	Inna Martin
Tel Nº:	+38 044 281 32 31
Email address of contact person:	imartin@unfpa.org

 

The deadline for question submission is Tuesday, November 27, 2018, at 16:00 Kyiv Time. All questions will be answered in writing and shared with all parties as soon as possible after this deadline.

 

3. Content of quotations

Quotations should be submitted in a single e-mail whenever possible, depending on file size. Quotations must contain:

1. Technical Proposal, in response to the requirements outlined in the service requirements / TORs. 
2. Financial Proposal, to be submitted strictly in accordance with the price quotation form.
3. Copies of registration documents.

The language of the proposal – English or Ukrainian. Separate Technical Proposal and Financial Proposal should be submitted in pdf format by the electronic method of transmission to the e-mail address indicated in section IV and be signed by the bidding company’s relevant authority.

4. Instructions for submission

Proposals should be prepared based on the guidelines set in TOR, along with a properly filled out and signed price quotation form, are to be sent to United Nations Population Fund (UNFPA) secured e-mail address to the contact person indicated below no later than: Thursday, November 29, 2018 at 17:00 Kyiv time. Proposals sent to any other address will not be considered.

 

 

Name of contact person at UNFPA:	Inna Martin
e-mail address of contact person:	ukraine.office@unfpa.org

 

Please note the following guidelines for submissions:

• The following reference must be included in the email subject line: RFQ Nº UNFPA/UKR/RFQ/18/12. Proposals that do not contain the correct email subject line may be overlooked by the procurement officer and therefore not considered.
• The total e-mail size may not exceed 20 MB (including e-mail body, encoded attachments and headers). Where the technical details are in large electronic files, it is recommended that these be sent separately before the deadline.

5. Award Criteria

UNFPA shall award a Contract with duration till December 31, 2018 to the company with the proposal received the panel's highest score. 

 

More information in the file attached.