fre 540 proposal (blue thunder)

29
540 Phase II M&E project.docx METRCompliant (M&E) Proposal for the MidTerm Evaluation of Livestock and Agricultural Marketing Project Mongolia Prepared by: Blue Thunder Consulting Xilun Zhang, 12328118 Financial, Technical Specialist Isaac Jonas, 87148145 Economist, Technical Specialist Submitted on Nov 30, 2014

Upload: xilun-zhang

Post on 14-Aug-2015

86 views

Category:

Documents


1 download

TRANSCRIPT

540  Phase  II  M&E  project.docx  METR-­‐Compliant  (M&E)  Proposal  for  the  Mid-­‐Term  Evaluation  of  Livestock  and  Agricultural  Marketing  Project  Mongolia  

   

   

Prepared by: Blue Thunder Consulting Xilun Zhang, 12328118 Financial, Technical Specialist Isaac Jonas, 87148145 Economist, Technical Specialist Submitted on Nov 30, 2014        

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 2

Executive  Summary  

  The  World  Bank  is  a  multilateral  organization  whose  mandate  is  to  reduce  poverty  by  providing   loans  and   investments,   facilitating  trade  and  supporting  sustainable  socio-­‐economic  projects   around   the   globe1.   Specifically   the   objectives   support   the  Millennium   Development  Goals   (MDGs).   The   World   Bank   approved   a   US$11.486   million   Livelihood   and   Agricultural  Marketing   project   (LAMP)   to   be   implemented   over   the   15   districts   of  Mongolia.   The   project  implementer  is  the  Ministry  of  Agriculture  Food  and  Light  Industry  (MOFLI).  The  project  will  last  over  four  years  since  May  17,  2013.  

 The  evaluators  will  conduct  the  Monitoring  and  Evaluation  for  Timely  Response  (METR)  

framework  to  achieve  the  listed  results:  • Collect   information   from   involved   stakeholders   (e.g.,   staff,   herders,   customers,  

government  official)  in  the  Livestock  and  Agricultural  Marketing  Project  (LAMP)  • Ascertain   successes,   challenges,   lessons   learn   and   timely-­‐respond   to   ensure  

project  outcomes  that  we  obtained  • Provide  METR  insights  to  help  the  project  stakeholders  achieve  the  set  of  goals  

 The  main  objectives  of  the  project  are:  

• To  improve  the  rural  livelihoods  • To  improve  food  security    

These  would  be  achieved  by  enhancing  productivity,  market  access  and  diversification  in  livestock-­‐based  production  systems.  

 

 

 

 

 

1 http://www.ehow.com/info_8106761_objectives-world-bank.html  

Table  of  Contents  

Introduction…………………………………………………………………………………………….5  METR  Resources  and  Methodology……………………………………………………………………………………………6                            METR  Skills  Involved  …………………………………………………………………….6     Team  Engagement  …………………………………………………………………………7     Project  Design  ………………………………………………………………………………11     Project  Implementation………………………………………………………………..14     Recommendations  ……………………………………………………………………….22  Works  Cited:  Project  Appraisal  Document  (PAD:  Report  Number  73827-­‐MN)  ……………………………………………………………………………………………………………….23  Appendix  A:  Sample  Questions  for  Stakeholder  Interviews…………………….24  Appendix  B:  Rapid  Logical  Framework  …………………………………………………...27  Appendix  C:  Problem  Tree  ……………………………………………………………………..29              

 

 

 

 

 

 

 

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 4

Acronyms  &  Abbreviations  

GAFSP-­‐Global  Agriculture  and  Food  Security  Program  GDP-­‐Gross  Domestic  Product  FAO-­‐Food  and  Agricultural  Organization  LAMP-­‐Livestock  and  Agricultural  Marketing  Project  Millennium  Development  Goals  (MDGs)  MSB-­‐Marginal  Social  Benefit    MEC-­‐Marginal  External  Costs  MOIA-­‐Ministry  of  Industry  and  Agriculture  Objective  Variable  Indicators  (OVI)  PAD-­‐Project  Appraisal  Document  TOT-­‐The  Trajectory  of  Trust  

 2  

                       

 

 

 

2 Definition  of  terms-­‐The  words  soum  and  aimags  are  used  to  mean  a  district  and  province  respectively  and  the  two  are  used  interchangeably  in  the  proposal

Introduction  

In  this  M&E  report,  the  evaluators  will  focus  on  figuring  out  the  loopholes  of  the  project  design  and  implementation  as  well  as  highlighting  the  potential  opportunities  for  the  project:        The   fall   of   the   Soviet   Union   left   Mongolia   at   crossroads   between   continuing   on   a   socialist  development  trajectory  or  taking  a  new  capitalist  direction.  Mongolia  chose  the  capitalist  route,  which   resulted   in   market   liberalization.   This   affected   the   agriculture   and   the   food   industry  negatively.  This  resulted  in  increased  rural-­‐urban  migration.        The  Mongolian   population   living   below   the   poverty   datum   line   increased   to   36.3   percent   in  1995   and   suddenly   dropped   to   29.8%   in   20113.Rural   poverty   increased   to   33%   for   the   same  period  (ibid).  The  slowdown  in  the  global  economy  adversely  affected  the  critical  drivers  of  the  economy,  which   are   agriculture,  mining   and   livestock  production.   The   crisis   limited   the   fiscal  space.   Until   2012,   there   still   exists   inequality,   poverty   and   food   insecurity   in  Mongolia   with  great  number  of  households  living  around  the  poverty  line.      Livestock-­‐based  agriculture  is  one  of  the  key  drivers  of  the  Mongolian  economy  accounting  for  at   least  85%  of   the  population.  The   livestock   industry  contributes  10%  of   the  country’s  Gross  Domestic  Product  (GDP).  The  Mongolian  GDP  fell  by  6%  and  has  a  population  of  approximately  3  million   people.   The   country   has   a   big   herd   of   about   20  million   herds   of   cattle,   18  million  sheep,  2  million  horses  and  300000  camels4.    The   Livestock   and   Agricultural   Marketing   Project   provides   advisory   services   to   herder  cooperatives  and  targets  5  declared  disease  free  zones,  which  are  correspondingly  Arkhangai,  Bayankhongor,  Govi-­‐Altai,  Khovsgul  and  Zavkhan.              

 

3http://www.worldbank.org/en/news/press-release/2013/05/21/poverty-rate-came-down-to-27-4-percent-in-2012 4http://www.encyclopedia.com/topic/Mongolia.aspx

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 6

METR  Resources  and  Methodology  

METR  Skills  Involved,  Team  Member  Responsibilities    Zhang  Xilun,  B.A.,  Economics,  MFRE  candidate  Role:  Financial  and  Technical  Specialist    Xilun  is  a  Master  of  Food  and  Resource  Economics  (MFRE)  candidate  at  the  University  of  British  Columbia  (UBC).  He  holds  a  Bachelor  of  Arts  degree  in  Economics  from  UBC.  His  research  interests  lie  in  the  field  of  agribusiness  and  commodity  trading.  He  has  strong  passion   and   practical   experience   in   financial   analysis   and   business   management.   His  financial   work   experience   is   drawn   from   his   stint   at   China   Citic   Bank   where   he   did  second  offering  and  FX  trading  for   Investment  Banking  Division  and  Financial  Markets.  He   gained   a   good   sense   of   the   foreign   currency  markets   as  well   as   customer   serving  skills.   Xilun   expects   to   integrate   the   current   MFRE   practical   skills   with   his   past  experience  from  finance  with  agribusiness.  He  is  an  avid  change  maker  desirous  to  apply  the  economics  background  and  financial  work  experience  to  the  real  world  development  puzzles.  

 Isaac  Jonas,  B.Sc.  in  Economics,  MFRE  candidate  Role:  Economist  and  Institutional  Specialist    Isaac   is   a  MasterCard   Foundation   Scholar   currently   studying   towards  Master   of   Food  and  Resources  Economics  (MFRE)  at  the  University  of  British  Columbia.  He  is  a  holder  of  a  bachelor  of  Science  Honors  degree  from  the  University  of  Zimbabwe.  Since  graduation  with   first   degree   in   2010,   he   has   been   working   as   a   development   practitioner   in  Zimbabwe.   He   has   worked   as   programs   manager   for   Youth   Career   Development  Zimbabwe  (YCDZ),  a  not-­‐for  profit  youth  development  organization  aimed  at  unlocking  career   opportunities   for   young  people   in   Zimbabwe  and   abroad.   Isaac   is   a   pragmatic,  dedicated  and  practical   change  agent.  He   is   a  Global   Youth  Ambassador   for  Award  at  School  -­‐  a  United  Nations  Education  Initiative  aimed  at  championing  access  to  education  for  the  approximately  57  million  young  people  around  the  globe  who  do  not  have  access  to  education.  He  brings  in  wealthy  of  institutional  experience  from  the  institutional  eye  view  based  on  his  experience  in  Zimbabwe.  

 

   

Team  Engagement    

Methodology  The  purpose  of  this  M&E  report  is  to  assess  the  project  performance  as  well  as  to  make  up   the   shortcomings   of   both   project   design   and   implementation   from   institutional,  economic   and   sustainability   perspectives.   The   two   students   in   the  MFRE   program   of  University  of  British  Columbia  will  work  as  a  team  “Blue  Thunder  Consulting”  will  do  the  project   assessment.   The   evaluators   will   assess   the   project   from   qualitative   and  quantitative  angles  under  the  criteria  as  indicated  below:    

• Relevance  of  Objective  • Efficacy  • Efficiency  • Sustainability  • Institutional  Development  Impact  • Impact  • Outcome  • Bank  Performance  • Borrower  Performance  

 The  methodology   includes   the   series   of   steps  needed   to   complete   the   final   report   by  May   31,   2015.Blue   Thunder   will   undertake   the   external   evaluation   based   on   the  methodology  listed  below:    1. Project  Launch  Meeting  

 The  Blue  Thunder  team  will  start  the  project  assessment  of  the  project  by  firstly,  getting  deep  understanding  of  the  PAD  and  information  beyond  the  PAD  because  it  basis  of  the  intervention.  This  is  really  important  to  get  involved  and  engaged  in  the  project  before  starting  the  evaluation  of  the  whole  project.  In  order  to  proceed  the  METR  process  more  efficiently   and   effectively.   The   team   decides   to   take   the   advantage   of   complimenting  expertise  of  the  two-­‐team  members.  The  institutional  specialist  had  been  working  as  a  development  practitioner  for  4  years.  Over  the  past  four  years,  he  has  accumulated  lots  of   social   and   institutional   work   experience   as   well   as   government   related   work  experience,  he  will  be  responsible  for  the  institutional,  governmental,  and  social  parts  in  our  M&E  Report.  Meanwhile,   Xilun   had   a   one-­‐year   investment   banking   experience   as  well  as  the  foreign  financial  markets  work  experience,  also  his  major   in  economics  will  

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 8

help   him   to   better   tackle   the   financial,   economics,   technological   parts   of   the   M&E  report.  

 Time  allocated:  1  day  

   2. Conduct  field  work  and  stakeholder  interviews    

 The   most   important   reason   to   do   field   work   while   doing   project   assessment   is   to  conduct   stakeholder   interviews   and   also   have   a   good   opportunity   to   ask   main  stakeholders  key  questions.  The  pilot  project  will  be  undertaken  in  5  aimags  (provinces).  The   five   aimags   are  Arkhangai,   Bayankhongor,   Govi-­‐Altai,   Khovsgul   and   Zavkhan.   The  Blue  Thunder  team  will  spend  15  days  to  visit  all   these  5  selected  provinces  to  have  a  detailed   observation.   Specifically,   the   team   will   do   30   one-­‐on-­‐one   stakeholder  interviews  and  5   interviews,  which   focus  on  stakeholder  groups.  During  that   time,   the  team  will  need  a  translator  to  communicate  with  the  stakeholders.  The  evaluators  will  examine  the  technology  in  three  ways  namely:      

(i) Enhancing  process  (ii) Improvement  results  for  post-­‐harvest  marketing    (iii) Stimulating  results  for  production  diversification  (iv) Invigorating  the  oversight  of  procurement  and  other  activities    

 Time  spent:  3  weeks  

 Possible  stakeholders  involved  

 • Key   beneficiaries   (e.g.   Herder   households,   marketing   agencies,   processing  

groups,  veterinarians  and  extension  workers).    o Herders  and  soum  residents  in  selected  aimags.  As  herders  are  producing  

most  of  milk,  meat,  fiber  and  they  operate  a  multi-­‐purpose  enterprise  for  Mongolia.  The  team  will  ask  to  get  their  views  about  the  project  as  there  may  be  some  issues  such  as  poor  animal  husbandry,  poor  market  access,  low  product  quality  and  poor  business  plan  development.  Specifically,  the  evaluators   would   get   more   information   about   the   training   process   for  those  extension  agents,  veterinarians,  herders  and  NGOs   involved  in  the  main  projects.  

 • Service  providers.  Blue  Thunder  Consulting  will  interview  some  service  providers  

who  are  responsible  for  delivering  the  advisory  services  to  the  selected  areas.  

During  the  interview,  the  evaluators  will  mainly  focus  on  the  problems  they  met  when  they  were  taking  their  responsibilities.  Specifically,  there  might  exist  emergencies  when  they  were  forming  development  of  business  plans.  For  example,  it  would  be  difficult  to  figure  out  whether  the  business  plan  is  financially  and  technically  feasible.  In  addition,  they  might  need  more  human,  physical,  social  and  financial  capital  while  linking  cooperatives  to  markets.  

 • Recipients  in  both  Sustainable  Livelihoods  Project  (SLP)  and  Index-­‐based  Livestock  

Insurance  Project  (e.g.  beneficiaries  selected  by  the  soum  social  welfare  office)      

• Private  processors.  As  the  project  will  encourage  commercial  buyers  to  participate  and  share  their  views,  the  evaluators  will  talk  to  those  private  processors/buyers  and  get  to  know  where  they  would  be  interested  in  the  project  implementation.  

 • Community  leaders  

 • Internal  evaluator  (e.g.  the  PIU  M&E  officer,  staff  in  MOAI)  

 • Main  donors  (e.g.  World  Bank  staff)  

 3. Synthesize  site  and  stakeholder  information  

 • The  reason  why  the  team  would  visit  sites  to  interview  stakeholders  is  to  help  fill  

gaps   in   both   information   and   related   knowledge.   After   the   evaluators   collect  useful   information   from   the   dialogue   of  main   stakeholders   and   observation   of  the   field,   the   team   will   come   up   with   ideas   and   recommendations   for   policy  change   to   address   their   needs.   These   would   be   useful   for   effecting   timely-­‐response  to  the  ongoing  LAMP.    

• Budget  and  Cost  Component:  o It  includes  the  transport  expense  travelling  to  the  selected  5  aimags  and  

living  expenses  during  the  time  we  do  the  field  work      Time  spent:  20  days  

 Timeline  

 

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 10

       

Project  Deliverables  • Deliverable  #1  -­‐  Draft  Report:  June  6,  2015  • Deliverable  #2  -­‐  Final  Report:  June  30,  2015  

 

 

 

 

 

 

 

 

 

 

 

 

 

Application  of  METR  Parameters  as  the  Basis  of  Performance  Assessment  

 Project  Design    

1. Conceptual  Integrity    • Assess  the  Logical  consistency  of  planned  interventions.  In  this  part  the  

team  will  figure  out  whether  the  objectives  of  the  project  are  consistent  with  the  objectives  of  the  organizations,  Ministry  of  Finance  and  Ministry  of  Food,  Agriculture  and  Light  Industry.  In  order  to  do  this,  the  evaluators  will  examine  the  PAD  to  see  if  there  is  any  difference.  

 2. Relevance  of  Objectives    

• The  assessment  mainly  focuses  on  the  relevance  of  the  project  objectives  as  well  as  how  well  they  are  going  to  be  reflected  in  the  project  design.  Specifically,  Blue  Thunder  Consulting  will   rate   the   relevance   in  a  one   to  five  scale.  The  objective   is  to   improve   livestock  productivity,  quality  and  strengthen  the  linkage  between  herders  and  markets.    

 3. Efficacy    

• In  this  part  the  evaluators  will  test  the  extent  of  the  project’s  objectives  achievement.  The  team  will  evaluate  the  relative  efficiency  of  the  scale  of  one  to  five  notches.      

 4. Logical  framework  

• The   project   did   not   have   its   own   logical   Framework   in   Project  Implementation,   which   includes   the   Objectives,   Objective   Variable  Indicators   (OVI),   Assumptions   and   Risks   and   Main   Activities.   The   team  created   the   Logical   Framework   using   information   of   Project   Appraisal  Document.   However,   because   of   the   lack   of   analytical   work   and  informational  gaps,  the  Logical  Framework  is  incomplete.  The  evaluators  will  work  with  project  implementers  to  solve  these  problems.  

     

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 12

5. Key  stakeholders  • Blue  Thunder  Consulting  will  visit  the  sites  and  stakeholders  in  person  to  

identify  the  property  rights,  responsibilities,  relationships  and  main  roles  for  all  the  main  stakeholders.  

 6. Assets  and  capital  management.    

• In   this  part   the   team  will   figure  out   the  most   important  assets   that   the  project  could  be  using  and  who  should  be  responsible  for  managing  these  assets.   The   team   will   mainly   focus   on   the   inefficient   allocation   of   the  assets  among  implementers  such  as  overlap  and  neglect  areas  existence  in   the   implementing   process.   Lastly,   the   evaluators   will   assess   the  following  capital  assets:  

 Financial  capital  (e.g.  grant  finance)  

Component  1:  • US  $6  million  grant  for  Living  Herders  to  Markets  • US  $4  million  grant  for  Meat  and  Fiber  Market  Development  • US  $1  million  grant  for  Dairy  Market  Development  • US  $1  million  grant  for  Pilot  Horticulture  Production  Component  2:  • US  $4  million  grant  for  Raising  Livestock  Productivity  and  Quality  • US  $1.5  million  grant  for  Promoting  Animal  health  • US  $1.5  million  grant  for  Animal  Breeding  and  Genetic  Improvement  • US  $1  million  grant  for  Animal  Nutrition  

    Component  3:  • US  $1  million  grant  for  Project  Management  

 Human  Capital.  Blue  Thunder  Consulting  will  take  human  capital  into  account  because  it  is  a  stock  of  productive  capabilities,  which  could  yield  a  flow  of  services  and  contribute  to  improve  productivity  in  Mongolia.  The  team  proposes  the  project  implementers  focus  on  investment  for  education,  health  system,  labor  training  and  skill  development  for  local  workers.  Specifically,  the  evaluators  will  first  compare  the  data  related  to  agricultural  commodity  production  before  and  after  the  project  is  undertaken  to  see  if  the  team  will  fulfill  the  project  objective  of  raising  livestock  productivity  and  quality.  The  evaluators  will  cooperate  with  the  officials  from  Government  of  Mongolia  (GOM),  Ministry  of  Industry  and  Agriculture  (MOIA),  Ministry  of  Information  System  (MIS)  and  Ministry  of  Finance  (MOF)  to  get  the  data  for  evaluation  and  analysis.  The  team  

will  do  financial  analysis  on  the  audit  reports,  balance  sheet  and  financial  statement  to  see  if  there  are  any  severe  problems  in  project,  company  and  association  records.    Secondly,  the  team  will  do  formal  surveys  such  as  household  income  survey  in  project  and  control  groups,  Core  Welfare  Indicators  Questionnaire  (CWIQ),  household’s  nutrition  survey  and  Client  Satisfaction  (or  Service  Delivery)  Surveys.  This  would  be  a  way  to  find  out  evidence  for  better  market  access  and  stronger  market  linkages.    

• Bank’s  important  role  in  the  livestock  sector    

• The  participation  of  Food  and  Agriculture  Organization  (FAO)  of  UN  (e.g.  field  a  team  of  expert  for  project  design  and  implementation)  

 • Key  role  of  the  International  Development  Associations  (IDA)  

rural  development  program    

Physical  Capital.  Blue  Thunder  Consulting  will  find  out  the  physical  capital  that  could  be  invested  in  the  selected  areas  in  Mongolia.  The  possible  physical  capitals  are  as  follows:  

• Infrastructure  (e.g.  roads,  transportation  equipment,  communication  line,  school)    

• Produced  capital  generated  by  applying  human  activities  that  aim  on  providing  goods  and  services  in  business  sectors,  public  sector  of  government  and  NGO  and  communities  in  Mongolia.  

 Social  Capital.  Blue  Thunder  Consulting  will  find  the  available  social  capital  because  it  is  a  stock  of  socially  held  knowledge  which  facilitates  the  social  coordination  of  economics  activities.  The  evaluators  will  find  the  characteristics  of  a  society  or  community  among  a  group  of  people  (e.g.,  workers)  whose  joint  work  will  contribute  to  efficiency  and  higher  productivity  of  Mongolia.  The  team  will  propose  a  survey  which  will  mainly  focus  on  the  social  activities  which  promote  people  to  work  together.  This  would  optimize  the  social  productivity  rather  than  maximize  individual  profits  at  the  expense  of  others.  Precisely,  the  team  will  conduct  well-­‐organized  events  to  do  cost  and  benefit  analysis  of  the  project.                  

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 14

Cost  and  Benefit  Analysis  (CBA).  Blue  Thunder  team  will  evaluate  the  opportunity  cost  of  the  12%  discount  rate  invested  in  Mongolia.    The  evaluators  will  analyze  the  opportunity  cost  of  capital  by  talking  to  the  project  implementers  and  looking  for  the  data  supporting  this  in  PAD.    

 Project  Implementation  

  Building  Inter-­‐Institutional  Trust  at  Inception    Government  Level  

 • The  government  of  Mongolia  plays  part  through  the  Ministry  of  

Industry  and  Agriculture  (MOIA)  and  Ministry  of  Finance  (MOF).    

Community  Level    

• The  beneficiaries  in  the  disease  free  provinces  (aimags)  of  Arkhangai,  Bayankhongor,   Govi-­‐Altai,   Khuvsgul   and   Zavkhan.   These   are   further  split   into   15   district   (soums).   The   target   population   herder   for   the  project   is   8110.   The   total   population   with   all   is   28385.   These   five  aimags  were  selected  on  the  basis  of  access  to  the  markets,  potential  fodder   production,   and   existence   if   herder   groups,   no   overlap  with  other   ongoing   development   projects   and   demand   for   value   chain  development.  There  will  be  80  extension  workers  who  will  benefit  as  agents  or  processes.  These  may  benefit  through  training.  

 • The  blue  Thunder  team  will  engage  a  carefully  chosen  sample  of  

beneficiaries  to  take  part  in  the  key  information  interview.  This  would  give  the  beneficiary  side  view  of  the  project.  

    Stakeholder  Analysis  and  Participation    

Donors  and  Government    

• As   indicated   above,   the   key   stakeholders   are   from   Government   of  Mongolia   (GOM)   as   represented   by   key   government   ministries  (Finance   &   Industry   and   Agriculture),   donors   like   Mercy   Corps,  Germany   Agency   for   International   Cooperation   (GIZ)   and   Non-­‐

governmental   organizations   (NGOs).   Food   and   Agricultural  Organization   (FAO)   would   provide   technical   assistance   for   food  related   activities   through   the   Global   Agriculture   and   Food   Security  Program   (GAFSP).  The  evaluators  will   contact   the  FAO   team  to  do  a  due  diligence  check  on  their  sustainability  of  their  support  on  a  rolling  basis.   The   impact   of   the   involvement   of   FAO   will   also   be   assessed  through   interviewing   the   beneficiaries   of   the   project   and   the   FAO  technical  staff.  

 Beneficiaries  

 • The   implementation   of   the   Livestock   and   Agricultural   Marketing  

Project  (LAMP)  might  pose  several  challenges  to  the  livelihoods  of  the  beneficiaries  in  Mongolia.  LAMP  may  create  a  power  struggle  for  the  community  between  those  community  leaders  of  the  project  and  the  ordinary  community  beneficiaries.  There  is  need  to  be  transparent  in  choosing   the   community   leaders   to   engage   in   the  whole   project   on  the   basis   of   merit   and   ability   to   integrate   the   community   to   drive  them   towards   one   common   goal   in   line   those   of   the   project.   This  would  improve  the  lives  of  the  de-­‐linked  Mongolian  herders.  

 • The   evaluators   would   also   look   into   how   much   the   project   would  

affect  welfare  of  the  children   in  the  process  of   involving  the  women  to  suit  gender  equity  and  equality  condition  of  the  project.  The  team  would   also   assess   the   hypothesis   that   women   headed   household  would   not   contribute   to   the   matching   grant   of   the   5   percent  contribution  in  cash  and  kind  (Project  Appraisal  Document  p.19)  

 • In  addition,  there  may  be  inconvenience  due  to  travel  time  between  

buyers   and   herders.   The   LAMP   could   potentially   disrupt   the  beneficiaries’  daily  routines  during  interviewing  and  survey  process  by  the  project  implementers.    

 • To  evaluate   these   inconveniences,   the  evaluators  will   conduct  a   site  

visit  across  the  five  project  areas  to  consult  on  the  most  efficient  and  effective   way   to   minimize   production   disruptions.   This   could   be  achieved  by  conducting  focus  group  interviews  with  the  stakeholders  of  the  project.    

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 16

   Program  Staff  

 • The   staff   from   involved   ministries-­‐Finance   and   Industry   and  

Agriculture,   National   Association   Agricultural   Cooperation   and   its  member   organizations,  Ministry   of   Social  Welfare,   SME   department  of  the  Ministry  of  Labor  would  constitute  the  local  project  staff.  There  should   be   a   critical   assessment   of   the   extent   these   project   staff  members  would   be   challenged  with   their   responsibilities   during   the  project   implementation  phase.  These  constitute   the  project   steering  committee.   It   is   critical   to   assess   and   strike   a   balance   between   the  different  staff  involved  in  the  project  and  avoid  over-­‐dependence  on  certain   staff.   This  would  enable  building  of   trust  between   the   locals  and   the   donor.   The   need   for   skills   training   and   knowledge   capacity  would  be  assessed.  Key  questions  like  motivation  and  rewards  will  be  raised  and  addressed.  

        Donor  and  Government      

• The   evaluators   will   conduct   a   due   diligence   assessment   on   the  present   market   conditions   across   the   horticulture,   livestock   and  animal   health   sectors   in   the   five   districts   of   Mongolia   where   the  project  would  be  implemented.  There  would  also  be  counterfactuals  to   spatially   and   temporally   check   the   variance   between   the   areas  where   the   project   is   being   implemented   and   non-­‐project   areas.   To  achieve   this,   the  evaluators  would   conduct   a   fact   finding  mission   to  study  the  market   linkages  and  synergies  across  the   five  provinces  of  Mongolia  where  the  project  is  taken  place.  The  evaluators  would  also  look   into   the  government   statistics   from  the  statistical  agencies  and  the   two   respective   ministries   involved   in   the   project.   The   use   of  Google  earth  image  would  also  be  applied  to  study  the  temporal  and  spatial  trends  across  the  areas  of  project  implementation.    

 Implementation  Governance  

 • The   fundamental   tenets   of   a   successful   project   hinge   on  

transparency,   accountability   and   information   sharing   across   the  board.   There   should   be   clear   responsibility   as   the   specific   duties  

assigned   to   key   stakeholders   and   those   who   are   answerable   when  things  go  off  track.  Respective  stakeholders  like  the  governments  and  donor   agencies   like   the   World   Bank   in   this   case   must   fulfill   these  necessary  conditions  for  project  success  to  be  realized.  

 • The   evaluators   would   scrutinize   the   financial   statements,   project  

documents  and  how  accessible  they  are  to  the  involved  stakeholders  and  the  public.  The  evaluators  will  also  look  into  the  preliminary  and  progress   reports   with   equal   attention   to   the   internal   monitoring  system  of  the  project.  The  team  would  also  engage  the  independent  think  tanks  and  follow  media  reports  on  the  project  as  a  signal  for  the  project  progress.  The  evaluators  would  round  off  by  interviewing  the  beneficiaries   from   across   the   board.   These   would   involve   the  livestock   and   horticulture   components   of   the   project.   The  institutional   staff   from   the   World   Bank,   Government   ministries   of  finance   and   Industry   and   Agriculture   and   the   community   leaders  would   also   be   contacted   regarding   their   willingness   to   share   their  views   on   the   progress,   challenges   and   opportunities   posed   by   the  project.  

 Implementation  Efficiency,  Effectiveness,  Legal  Framework  and  government  policy  

 Efficiency    

• To  measure  efficient  use  of  time  resource  during  the  project  life  cycle,  the   evaluators   will   engage   the   key   beneficiaries   involved   in   the  livestock   and   horticulture   components   of   the   project.   This   will   be  achieved  by  interviewing  them  their  views  on  the  project.  

 • The   evaluators   will   measure   the   opportunity   cost   of   labor   even   in  

cases   where   there   is   ‘free’   labor   from   the   beneficiaries   through  volunteers.  This  would  answer  the  question  of  the  true  market  cost  of  labor  even  in  case  there  were  volunteers  who  assist  in  the  project.    

 • To   check   on   the   proportion   of   administrative   costs   to   the   whole  

project  activities,  the  evaluators  will  find  out  the  ratio  of  the  two  and  comparatively   analyze   the   financial   resources   committed   to   the  project   activities   as   in   relation   to   the   administrative   expenses.   The  issue   of   corruption   and   or   theft   would   also   be   assessed   by   key  

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 18

informant   interviews   with   the   beneficiaries   across   the   five   project  areas.   The   evaluators   would   also   check   out   with   local   newspapers  whether   there   have   been   cases   of   theft   or  misappropriation   of   the  project  money.  

 • There  shall  also  be  engagement  of  external  environmental  analysts  to  

get   the   independent   outsider’s   view   with   regards   to   the   project  impact.  This  would  be  done  transparently  by  publicly  advertising  and  selecting  the  best  applicant  based  on  meritocracy.  

 Effectiveness    

 • The  evaluators  will  conduct  a  diligence  check  on  whether  the  project  

is   on   a   successful   path.   This   would   be   accomplished   by   making   a  comparison  between  the  set  out  project  goals  and  what  actually  take  place   on   the   ground   across   the   five   project   areas.   Blue   Thunder  Consulting   would   assess   the   impact   of   excluding   female-­‐headed  household   in  making   financial   contributions   on   the   project.  We  will  interview   the   beneficiaries   on   this   key   issue   as   it   has   potential   for  creating   divisions   between   those  who   are   financially   better   off   and  financially   poor.   Furthermore,   the   evaluators  would   assess  whether  having   the   male-­‐headed   household   beneficiaries   contributions   did  not  pose  a  burden  due  of  different  income  contribution  abilities.  

 • The   evaluators   will   examine   the   linkages   between   the   different  

producer   groups   and   commodity   associations   to   investigate   the  linkages   between   the   two.   This   is   done   to   evaluate   whether  bargaining,   as  a  union   is   the  best  way   to   create  market   linkages   for  the   project.   The   role   of   involved   intermediary  National   associations  such  as  the  Dairy  Breeder  Association  would  also  be  evaluated.  In  this  case,   the   evaluators   would   interview   key   informants   from   the  beneficiaries  and  the  respective  national  associations  to  get  their  side  of   the   story   with   regards   to   this   issue.   The   evaluators   will   also  research   from   the   local   media   about   these   organizations’   conduct.  This  is  because  they  play  a  key  role  in  LAMP.  

 • The  evaluators  would  also  look  into  the  ex  ante  and  ex  post  analysis  

to   see   the   difference   and   progress   on   creating   the  market   linkages  and  on  the  counterfactual  areas.  

 Impact  

 • The  impact  of  a  project  answers  the  question  of  the  extent  to  which  

the   project   improves   the   livelihoods   of   the   intended   beneficiaries  (World   Bank).   This   could   be  measured   by   analyzing   the   variance   in  incomes   between   the   project   areas   compared   to   the   non-­‐project  areas  (counterfactuals).  The  evaluators  will  compare  the  incomes  for  the   herders   from   a   sample   of   five   provinces   where   LAMP   was  undertaken.   This   would   be   analyzed   and   compared   to   the   income  levels   of   the   surrounding   community   where   the   project   was   not  implemented.  Further,  across  the  sample  of  the  15  districts  where  the  project   is   being   implemented,   the   team   will   also   further   split   the  herders  into  two  categories  i.e.    

1. Income  for  early  adopters  2. Late  adopters  

• These   would   be   analyzed   and   compared.   Blue   Thunder   Consulting  would  expect  on  average,  ceteris  paribus,  the  early  adopters  to  have  higher  income  streams  compared  to  the  late  adopters  if  the  project  is  benefiting   the   intended   beneficiaries.   The   team   would   expect   the  early  adopters  to  reap  more  benefits  from  early  connections  to  more  lucrative  export  markets.  

 • In   light   of   the   above   point,   the   evaluators   will   also   document   and  

analyze   the   statistics   of   those   people   who   would   be   involved   in  forward   contracts   with   respective   buyers   of   livestock   products   and  horticultural   products.   The   team   would   expect,   on   average,   an  increasing   rate   of   the   herders   and   horticulturalists   in   project   areas  entering   into   the   futures   market   contracts   compared   to   the   non-­‐project  areas.  This  would  be  an  indication  of  better  market  linkages  in  LAMP  areas  (Purcell  and  Hudso:  1985)    

 • The  Blue  Thunder   team  will   interview  a   sample  of  beneficiaries   and  

the   project   surrounding   area   population   about   their   income  experience.   The   team  would   also   conduct   research   on   the   Internet  and  other  local  media  houses  to  track  down  the  income  changes  that  may  be  accruing  from  LAMP.  

 

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 20

• The  second  methodology  would  be  to  use  light  illuminating  evidence  to   track  down   the  development   in   the  project   implementation   zone  against  the  surrounding  areas.  On  average,  all  things  being  equal,  the  team  would  expect  more   indication  of  development  as   indicated  by  more   illuminating   light   in   areas   where   the   project   is   being  implemented  than  in  non-­‐project  areas  (rural  areas  alike)  

 • The  Blue  Thunder  team  would  also  enumerate  the  differences  in  the  

livestock  herds   growth  and  horticultural   yield  per   acre  between   the  project   and   non-­‐project   areas.   All   things   being   equal,   we   would  expect   the   LAMP  project   areas   to   have   higher   scores   in   all   the   two  indicator   scores   compared   to   the   non-­‐project   areas.   The   research  impact   would   be   achieved   by   gathering   and   comparing   agricultural  statistics  across  the  15  project  areas  against  the  non-­‐project  areas.  

 The  Trajectory  of  Trust  (TOT)  

 • The   project   involved   partnership   between   the   agribusiness   and  

producers   and   also   incorporated   the   governor’s   offices   at   all   levels.  This  is  a  crucial  principle  to  build  mutual  respect  that  is  important  to  build  trust.  Locals  serve  as  the  face  for  the  other  beneficiaries  as  they  would  have  a  sense  of  ownership  of  the  program.  

 Environmental  Assessment  and  Social  Safeguards  

 • Environment   is   defined   to   mean   the   surroundings   in   the   project  

areas.   This  may  be   the  human  and  or  physical  environment   like   the  flora   and   fauna.   The   environmental   assessment   standards   were  conducted   according   to   the  World   Bank  Operational   Policy  OP   4.01  standards.   This   assesses   the   negative   externalities   posed   by  implementing  the  project.  As  a  guideline,  the  Marginal  Social  Benefit  should   exceed   the  Marginal   External   Costs   (MEC)   of   implementing  the  project.  The  team  will  interview  the  beneficiaries  on  the  impact  of  the  project  across  the  15  districts.  The  evaluators  will   take  a  sample  for   each   province   to   ask   questionnaires   about   the   environmental  impact   of   the   project.   Extensive   research   on   the   media   about   the  environmental   impacts   of   the   project   would   also   be   tabled.   The  evaluators   will   follow   key   environmental   news   and   trends   on   the  

international  media  with   particular   focus   on  Mongolia   or   East   Asia.  Although   costs   are   not   easily   measurable,   the   team   will   use   the  closest   approximation   methodology   to   come   up   with   a   fair   cost  estimate.  

 • To  ensure  this  crucial  principle  would  be  followed,  the  evaluators  will  

conduct   interviews   with   the   beneficiaries   to   the   extent   of   the  selection  of  the  local  project  partners.  

 Government  policy  

 • Government   policy   plays   a   key   role   in   national   development.   The  

evaluators  would  assess  the  extent  to  which  LAMP  adhers  to  the  laws  of  Mongolia.   This   is   important   to  make   sure   the  project   follows   the  law   otherwise   it   would   be   a   constraint   to   the   project   in   both  intermediate  and  long  term.  

                                           

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 22

Recommendations  to  improve  the  project  result  in  terms  of  all  aspects  of  the  project  The  M&E  recommendations  will  focus  on  the  following  aspects  of  the  project  design  and  implementation:  

• Technology  introduction  and  usage  • Capital  allocation  and  management  • Project  sustainability  • Economic  efficiency  • Project-­‐support  training    

 In  order  to  enhance  project  design  and  implementation,  the  evaluators  will  provide  recommendations   in   terms   of   project   efficiency,   efficacy,   sustainability,   level   of  management  and  capital  allocation.  Most  importantly,  the  most  severe  issues  in  the  livestock   sector   in   Mongolia   is   the   poverty   and   low   productivity,   the   team   will  research   beyond   the   PAD   document   and   find   out   solutions   to   improve   project  efficiency  if  need  be.  

                                 

 

 

Referenced  Material  

 “Marginal   Social   Cost   (MSC)   Definition.”   Investopedia.   Accessed   November   14,   2014.  http://www.investopedia.com/terms/m/marginalsocialcost.asp.  

 Mikoluk,  Kasia.  “Marginal  Social  Benefit:  Basics  of  Microeconomics.”  Accessed  November  14,  

2014.  https://www.udemy.com/blog/marginal-­‐social-­‐benefit/.    http://www.worldbank.org/projects/P122123/zambia-­‐livestock-­‐development-­‐animal-­‐health-­‐

project?lang=en  .  Accessed  November  17,2014.    http://www.gafspfund.org/sites/gafspfund.org/files/Documents/Mongolia_LAMP_IE_Conce

pt%20Note_final.pdf  .  Accessed  November  17,2014    http://www.farmdoc.illinois.edu/irwin/archive/books/Futures-­‐Regulatory/Futures-­‐

Regulatory_chapter6.pdf  .  Accessed  November  17,2014  http://www.ehow.com/info_8106761_objectives-­‐world-­‐bank.html.Accessed   online.29  

November  2014.    

                                 

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 24

Appendix  A  –  Sample  Questions  for  Stakeholder  Interviews                

Partners    World  Bank  

1. Why  did  you  provide  funding  for  the  Livestock  and  Agricultural  Marketing  project?  2. What  are  the  goals  of  the  Livestock  and  Agricultural  Marketing  project?  3. Do   you   think   the   Livestock   and   Agricultural   Marketing   project   have   done   what   you  

expect  so  far  this  year?  4. Have  you  visit  that  15  selected  areas  of  the  project?  5. Do  you  have  dialogues  with  the  beneficiaries?  (e.g.,  Herders)    6. What   do   the   community   members   comment   on   the   Livestock   and   Agricultural  

Marketing  Project?  7. What  do  you  think  is  the  major  flaw  of  the  project?  8. In  which   aspects   do   you   think   the   Livestock   and   Agricultural  Marketing   Project   could  

improve?  How  to  improve  it?    Project  Developers  and  Implementers  Ministry  of  Food,  Agriculture  and  Light  Industry  

1. What  is  the  best  moment  during  the  project  development  step?  2. What  were   your  most   original   objectives   for   the   Livestock   and  Agricultural  Marketing  

Project?  3. What  do  you  think  the  Livestock  and  Agricultural  Marketing  Project  could  improve?    4. Are  there  some  big  issues  during  the  project  process  that  need  to  be  fixed?  5. What  are  some  big  challenges  of  Livestock  and  Agricultural  Marketing  Project  in  steps  of  

Designing,  Planning,  Developing  and  Implementing?  6. Did   you   have   any   problems   or   barriers   while   developing   the   project?   (e.g.,   finance,  

support  of  the  government)  7. Who  received  training  during  the  project  process?  8. Did  you  receive  grant  funding  last  year?    9. How  much  grant  funding  did  you  received  last  year  for  the  project?  10. Does  the  grant  funding  amount  meet  the  amount  that  you  expected?  11. Are  there  any  changes  in  the  neighborhoods  after  the  project  started?  12. What   do   you   hear   about   the   comments   on   the   Livestock   and   Agricultural  Marketing  

Project  from  community  members?  13. Would  you  recommend  the  Livestock  and  Agricultural  Marketing  Project  to  other  areas  

similar  to  Mongolia?  Why  and  why  not?    

Beneficiaries  Herders  from  selected  areas    

1. Do  you  think  the  Livestock  and  Agricultural  Marketing  Project  really  change  your  lives?  2. Did  the  Livestock  and  Agricultural  Marketing  Project  increase  your  income?  3. Do  you  have  any  recommendations  on  the  project?  4. Do  you  think  you  are  better  off  in  your  lives  than  other  herders  that  are  not  included  in  

the  project?  5. What   aspects   do   you   think   the   Livestock   and   Agricultural   Marketing   Project   should  

change  to  be  more  efficient?  6. Do   you   think   the   Livestock   and   Agricultural   Marketing   Project   changed   your  

neighborhood?  Make  it  better  or  worse?    Staff  The  PIU  M&E  officer  and  staff  in  MOAI  

1. Do  you  have  any   related  project-­‐related  work  experience  before   joining   the   Livestock  and  Agricultural  Marketing  Project?  

2. What  do  you  think  were  the  hardest  parts   in   the  Livestock  and  Agricultural  Marketing  Project?  

3. What  good  habit  and  skills  have  you  cultivated  during  the  whole  process  of  the  project?  4. Do  you  think  the  project  will  achieve  its  goals  or  meet  its  objectives  during  the  process?  5. What  do  you  think  the  project  components  should  add  in?  6. Do   you   think   the   objectives   of   the   project   are   consistent   with   the   objectives   of   the  

organization  helping  to  implement  the  Livestock  and  Agricultural  Marketing  Project?  7. Do   you   think   the   herders   will   really   benefit   from   the   Livestock   and   Agricultural  

Marketing  Project?  8. To   what   extent   do   you   think   the   sustainability   of   the   Livestock   and   Agricultural  

Marketing  Project  will  have?    Community  members  

1. What  do  you  think  the  Livestock  and  Agricultural  Marketing  Project  has  created  for  the  community?  

2. What   has   been   changed   to   the   neighborhood   after   the   Livestock   and   Agricultural  Marketing  Project  got  started?  

3. What  did  the  neighborhood  look  like  before  the  project  implementation?  4. What  did  the  neighborhood  look  like  after  the  implementation?  5. Do   you   think   the   Livestock   and  Agriculture  Marketing   Project   has   brought   benefits   to  

the  community?  

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 26

6. What   do   you   think   the   Livestock   and   Marketing   Project   should   improve?   To   what  aspects  it  could  be  improving?  

                                                         

   

 

Appendix  B  –  Rapid  Logical  Framework  

                                                   

               

Proposal – METR-Compliant Mid-Term Review, LAMP-Mongolia 28

           

 

                     

 

 

 

Appendix  C  –  Tree  Diagram