Chapter 6 The Monitoring and Evaluation of Outcomes

Chapter 6 The Monitoring and Evaluation of Outcomes

Current Level of Engagement
Issues and Conclusions
Objectives
Benchmarks and Indicators
Other Measures of Success
International Conferences

Current Level of Engagement

6.1

The DFAT submission states that:

“Monitoring and evaluation of the dialogues takes place in the context of our assessment of our broader engagement with dialogue partners on human rights issues.”1

6.2

DFAT monitors and evaluates the dialogues according to:

6.3

DFAT notes that there are difficulties inherent in assessing any direct impact that Australia’s bilateral human rights dialogues has on improving the human rights situation in a particular country:

“…we are realistic about the significance of our dialogues, in and of themselves, as levers for fundamental or rapid change. The process of change is generally incremental, and is the result of a range of contributing factors, of which our bilateral dialogues are one.”3

 

Issues and Conclusions

6.4

Several submissions to the inquiry expressed concern that the current measures for monitoring and evaluating Australia’s human rights dialogues were too general and this detracted from the transparency, accountability and credibility of the dialogues. Suggestions for more specific and/or outcome-oriented measures discussed below include:

6.5

The Committee discussed these issues at the public hearing.

Objectives

6.6

At the hearing, the Committee asked DFAT if the human rights dialogues had clearly defined objectives and focused evaluation criteria. DFAT replied that whilst there were no specific benchmarks, there was certainly a broad purpose (namely to discuss human rights issues) and implicit shared objectives in the agenda.7

6.7

HREOC told the Committee that the technical cooperation program has clearly stated objectives and the Commission evaluates every activity against those objectives and reports on them.8 The HREOC submission contains evaluation-related extracts from Commission reports for each of the dialogue partner countries, China, Vietnam and Iran.9

 

Benchmarks and Indicators

6.8

In written evidence, ACFID refers to the EU guidelines on human rights dialogues as a model for benchmarking the progress of human rights dialogues in order to make the process more accountable. According to ACFID, international NGOs, including Human Rights Watch and the International Federation for Human Rights (FIDH), have developed similar indicators for measuring a dialogue country’s commitment to achieving human rights outcomes. The indicators comprise:

6.9

At the hearing, ACFID reiterated its concerns that Australia did not use benchmarks to measure demonstrable progress in advancing human rights in dialogue partner countries:

“If you cannot demonstrate that you are making some progress – and it does not have to be fantastic achievement; baby steps are sufficient – and that the dialogue is making a positive contribution, it raises the question of what point there is in having a dialogue. Is it merely window-dressing?”11

6.10

The Committee was interested to learn more about the EU Guidelines and other indicators and how those evaluation criteria might be applied to Australia’s human rights dialogue process to enhance successful outcomes. Both ACFID and Amnesty International offered to provide the Committee with that level of detail later.

6.11

Subsequent to the hearing, both ACFID and Amnesty submitted a copy of the EU guidelines on human rights which were prepared by the Council of the EU in December 2001.12 Appendix E contains a copy of the guidelines.

6.12

In summary, the EU document offers guiding principles for the conduct of human rights dialogues. Examples of the guiding principles include:

6.13

Both ACFID and Amnesty also supplied a copy of the Council of the European Union’s benchmarks in respect of the EU-China human rights dialogue (from the FIDH report which ACFID referred to at the hearing). Appendix F contains a copy of the document.

6.14

These indicators, like the EU’s guidelines, are quite broad. Some examples include:

6.15

It is the Committee’s view that both sets of EU guidelines provide general guiding principles rather than specific criteria against which to measure the progress and/or success of the dialogues. Neither set indicates how to use the criteria. For instance, should ratification and implementation of the international human rights treaties be a precondition for talks or a goal to work towards? This is one of the inherent challenges of creating meaningful benchmarks and indicators.

6.16

Another challenge is to avoid using indicators dogmatically. The European Commission cautioned the EU against using indicators in this manner:

“…the EU should avoid the mechanistic use of indicators, or attempt to compare or rank countries’ performances. Each situation is different. Trends matter more than snapshots.”15

6.17

In any case, the principles set out in the EU’s guidelines have broad parallels with the assessment criteria that Australia uses to measure the progress of its dialogues as outlined in the DFAT submission.16 Measures such as monitoring progress on individual cases of concern, consulting with civil society, and evaluating the degree of openness and level of engagement with partners have a similar ethos to the EU’s measures.

6.18

It is difficult for the Committee to judge from the evidence given how the EU’s guidelines might be applied to the Australian situation – other than as general guiding principles – which are, in the main, already in place. Further, while some submissions, including those from ACFID and Amnesty, advocate that benchmarks should be used as evaluating criteria for the dialogues, other submissions to the inquiry remained silent on the subject.

 

Other Measures of Success

6.19

Given NGO concerns about the need for benchmarks and indicators as evaluating criteria for the dialogues, the Committee asked agencies what measures they use to evaluate the dialogue process.

6.20

DFAT told the Committee that the Government does not use formal benchmarks in its monitoring and evaluation of the dialogues, nor does it intend to establish any at this stage:

“I think that it is correct to say that the government does not consider that having formal benchmarks is necessarily very useful, particularly in a situation where change in the countries concerned is often incremental. There are often a number of factors that result in that change, of which our dialogue is a part…It is quite difficult to be able to measure precisely - we are not sure that it would be very productive - what has been done directly only as a result of the Australian dialogue. I do not see that there is any particular move to go along those lines at this stage.”17

6.21

The Department said that it prefers to take a holistic approach and use the following evaluation criteria, amongst others, to monitor the dialogues:

“In terms of monitoring the effect of the dialogue, clearly there are factors that we would look at. One of those would be progress in

individual cases of concern. We also look at the overall human rights situation in the country concerned.”18

“Things like progress in the legal system, how human rights are being dealt with, specific changes to legislation and the way legislation is being implemented are things you can monitor and evaluate…[also] the release of prisoners and the treatment of prisoners are all quite specific issues that we look at as part of trying to evaluate progress on human rights.”19

6.22

The Committee questioned DFAT on how, in taking this approach, it determined the success or otherwise of the dialogues. The Department explained that it makes a value judgement based on a number of contributing factors. The degree of openness, trust and willingness to engage are important elements.20

Australia - China Dialogue

6.23

DFAT stated that in the case of Australia’s longest-standing dialogue, with China, the exchange has, over the years, evolved to become a more frank dialogue:

“Rather than getting a set piece response, getting a much more informed response and a willingness to admit that there are issues that need dealing with and to talk about the government in China is doing to address them…it was much harder to have that discussion some years ago.”21

6.24

HREOC added that it takes time to build cooperative bilateral relationships. For instance HREOC has wanted to work in Tibet for some years but it has taken until quite recently for that to come to fruition:

“It has taken us some time to establish enough confidence with the authorities in the Tibetan autonomous region to allow us to work there. We now have an initial activity that will take place in July next year, involving the delivery of minority language education in Tibet.”22

6.25 The Committee took other evidence to suggest that the Australia - China dialogue is characterised by an increased openness, trust and willingness to engage on a range of human rights issues. The Attorney-General’s Department described some positive exchanges that have come out of more recent China dialogues, including discussions about the practicalities of administrative appeals and administrative review legislation and independently of the dialogue sessions, discussions with civil society:

“When I was in China, [we had a] meeting with the All-China Women’s Federation…about all the protections and legislative prohibitions on the sale and trafficking of women.”23

6.26

Earlier in the hearing, ACFID itself described how for the first time, at last year’s Australia - China dialogue, five Australian NGOs had met with Chinese officials. ACFID indicated they had had a fruitful discussion about various human rights issues (see Chapter 3). This dialogue was also the first occasion at which a press conference was held.

6.27

HREOC noted that one of the successes of the Australia - China dialogue was that Australia continues to be one of the partners of choice for cooperation in the most sensitive of human rights areas:

“It is perhaps notable that one of the aspects of the reform agenda of the National Population and Family Planning Commission is specifically to seek out cooperation with Australia in the human rights aspects of family planning.”24

6.28

HREOC cautioned against focusing too intently on assessing Australia’s direct impact on the human rights situation in China via the dialogues. The Commission stated that the purpose of human rights technical cooperation is to expose Chinese agencies to Australian expertise and experience in the promotion and protection of human rights, but that it is ultimately up to Chinese agencies to inform and develop their own reforms:

“Let me be unequivocal about this: all human rights reforms in China result from conclusions drawn by the Chinese and from actions taken by them themselves.”25

Australia - Vietnam and Australia - Iran Dialogues

6.29

At the hearing, the Committee asked the participating agencies for updates on the lesser established Vietnam and Iran human rights dialogues. HREOC said that it had conducted an initial planning mission in Vietnam recently, in January 2005, with the intention of establishing a technical cooperation program there. To date, there have been no moves to establish a similar program with Iran.26

6.30

The Committee asked DFAT to elaborate on the present human rights situation in Iran, whether it had deteriorated in recent times, and if so, what ramifications, if any, there were for bilateral human rights dialogues with Iran to continue. The Department said that there were people in Iran who remain committed to reform and working on the human rights situation in Iran and that the Australian position was to remain engaged:

“We see it that if we backed off now it would be interpreted as the Australian government not caring about the human rights situation in Iran. There is this lack of international interest, so we see that we really need to maintain it….We have been keen to get a second round [of dialogues] going and to continue to talk to them”27

 

International Conferences

6.31

In written evidence, Amnesty suggested that Australia should convene –possibly in conjunction with the United States or the European Union – an international meeting of dialogue countries for the purposes of exchanging information on and approaches to dialogue processes.28

6.32

At the hearing, the Committee questioned DFAT about the extent to which Australia engaged with other countries that conduct bilateral human rights dialogues to discuss the dialogue process, and asked what international fora exist for dialogue countries to exchange experiences.

6.33

The Department indicated that Australia communicates informally with the Europeans on human rights dialogue matters. DFAT said that:

“We are very keen to talk even more closely and formally with the Europeans on some of this. If there were prospects for doing something more broadly I think we would look at it…”29

6.34

DFAT named two international meeting forums which officials from countries that conduct human rights dialogues with China and Iran attend, to exchange information and experiences: the Berne and Brussels-Berne processes, respectively.30

6.35

DFAT later confirmed Australia’s participation in both these forums, stating that representatives from the nearest Australian mission attend them.31

6.36

HREOC told the Committee that it also engages in information exchange activities with a range of national and international agencies in respect of the technical cooperation aspects of the dialogue process.32

 

Senator A B Ferguson

Chair

12 September 2005



Footnotes

1 Submission no. 17, DFAT, p. 12 Back
2 Submission no. 17, DFAT, p. 12 Back
3 Submission no. 17, DFAT, p. 12 Back
4 Submission no. 6, ACFID, p. 12 Back
5 Submission no. 4, Australian Baha’i Community, p. 5, Submission no. 5, Joint Non-Government Organisations, p. 1 & Submission no. 6, ACFID, p. 12 Back
6 Submission no. 8, Amnesty, p. 10 Back
7 Official Transcript of Evidence, DFAT, p. 41 Back
8 Official Transcript of Evidence, HREOC, p. 18 Back
9 See Submission no. 14, HREOC, Attachments 2-4 Back
10 Submission no. 6, ACFID, p. 11 Back
11 Official Transcript of Evidence, ACFID, p. 7 Back
12 Submission no. 19, Amnesty, p. 12 & Submission no. 20, ACFID, p. 3 Back
13 See Appendix E for full document Back
14 See Appendix F for full document Back
15 European Commission, Communication from the European Commission to the Council and the European Parliament, Brussels , May 2001 Back
16 Submission no. 17, DFAT, pp. 12-16 Back
17 Official Transcript of Evidence, DFAT, p. 40 and p. 46 Back
18 Official Transcript of Evidence, DFAT, p. 41 Back
19 Official Transcript of Evidence, DFAT, p. 45 Back
20 Official Transcript of Evidence, DFAT, pp. 44- 45 Back
21 Official Transcript of Evidence, DFAT, p. 45 Back
22 Official Transcript of Evidence, HREOC, p. 23 Back
23 Official Transcript of Evidence, Attorney-General’s Department, p. 52 Back
24 Official Transcript of Evidence, HREOC, p. 16 Back
25 Official Transcript of Evidence, HREOC, p. 16 Back
26 Official Transcript of Evidence, HREOC, p. 19 Back
27 Official Transcript of Evidence, DFAT, p. 50- 51 Back
28 Submission no. 8, Amnesty, p. 10 Back
29 Official Transcript of Evidence, DFAT, p. 53 Back
30 Official Transcript of Evidence, DFAT, p. 53 Back
31 Exhibit no. 7, DFAT Back
32 Official Transcript of Evidence, HREOC, p. 19 Back

Print Chapter 6 (PDF 127KB) < - Report Home < - Chapter 5 : Appendix A - >