Yeah! We have open data

Last April 30th was the first milestone for publishing an open dataset for all partners in the “Dialogue & Dissent” strategic partnerships.
So what happened?

Over the last couple of weeks there was a steady rise in the number of Dutch IATI publishers. Bill Anderson (of the IATI secretariat) even tweeted:


In the past 2 weeks I’ve looked at all the datasets to see what’s in there: “who are the organisations who have published?”, “how complete are the data?” and of course “can these data be used to show networks of activities?” – the main focus of the guiding principles we developed.

Of all the partners in the strategic partnerships (65, including the ministry), the majority (40) have published an IATI dataset that includes the partnership activities. Then there were a few cases where the organisation did publish a dataset, but I couldn’t find the partnership activities. Two files wouldn’t download. Other organisations are still working towards their first open data publication. Overall I find this a pretty positive picture!


Looking at the content of the data sets I got even more enthusiastic: it’s clear that most organisations made an effort to publish pretty complete data sets using many data elements ánd using the elements that help creating networks: organisation and activity identifiers and referring to related activities. And even though the work on the strategic partnerships is just underway, many organisations published meaningful descriptions of their planned activities, included links to documents or started to work on their result framework in IATI format.

I’ll highlight a few things that struck my eye while going the files:

The use of identifiers is apparently still a bit unclear. Although we’ve tried to make the text in the guidelines as clear as possible and IATI gives a thorough explanation too, this is the subject that we’ve gotten most questions about. And looking at the published IATI files it isn’t always implemented correctly.

We’ll go through the guidelines once more, to update texts that are still unclear. If you have specific feedback, please get in touch!

In the meantime let me work out another example of two organisations referring to each other: Ministry of Foreign Affairs (DGIS) and Both Ends.

Both Ends is the lead organisation in the Fair Green and Global Alliance partnership, and is funded by DGIS.

In the DGIS administration the partnership with Both Ends is supported by activity PPR-27534. In DGIS’s dataset you’ll find this activity with number NL-1-PPR-27534 and title “SP-S&T/Both Ends”.
DGIS refers to Both Ends in the <participating-org> element:

<participating-org ref="22000" type="22" role="4">
<narrative xml:lang="en">STICHTING BOTH ENDS</narrative>

and also in the transactions, for instance in the commitment:

<transaction-type code="2">
<transaction-date iso-date="2015-11-23">
<value value-date="2015-11-23">59523750</value>
<provider-org ref="NL-1">
<narrative>Ministry of Foreign Affairs (DGIS)</narrative>
<receiver-org ref="22000">
<narrative>STICHTING BOTH ENDS</narrative>

Notice that DGIS doesn’t use the correct organisation identifier for Both Ends yet. Instead of “22000” it should use “NL-KVK-41210098”. (“22000” is a general OECD-DAC code for “Donor country-based NGO”)

In Both Ends’ administration the strategic partnership program is identified as FGG and title “Fair, Green and Global Alliance”.
In its IATI dataset Both Ends publishes the activity with number NL-KVK-41210098-FGG.
Both Ends refers to DGIS in the <participating-org> element as the funder of the activity:

<participating-org ref="XM-DAC-7" type="10" role="1" activity-id="NL-1-PPR-27534">
<narrative>Ministry of Foreign Affairs Netherlands</narrative>

and also in the incoming fund transactions:

<transaction ref="FGG-1">
<transaction-type code="1"/>
<transaction-date iso-date="2015-12-15"/>
<value currency="EUR" value-date="2015-12-15">5952375</value>
<provider-org ref="XM-DAC-7" provider-activity-id="NL-1-PPR-27534"/>
<receiver-org ref="NL-KVK-41210098" receiver-activity-id="NL-KVK-41210098-FGG"/>
<tied-status code="3"/>

Notice that Both Ends uses an incoming funds transaction (transaction-type code="1") and is not using the incoming commitment yet (transaction-type code="11")

Segmented files
A second thing that I notices, is that quite a few organisations published so-called segmented files: a number of files each containing 1 or 2 activities in specific countries, rather than publishing all the activities in one ‘unsegmented’ file.

In general, splitting your data set (although advised by IATI) is not a great idea: when the data are divided over multiple files, it’s harder to bring them together again and analyze the content. Currently the most used segmentation is per country/region and this introduces another problem: how do you deal with an activity that takes place in two countries? Do you split up the activity into 2 smaller ones? Or should you then define a region, just to be able to publish the activity?

The most probable reason so many organisations publish segmented files is a default setting in Aidstream – and it is easy to fix:

  • When you log into Aidstream, look at the menu on the left hand side, scroll down and choose
  • Then scroll down to “Publishing type” and choose “Unsegmented”.

Whenever an activity aims at more than one sector, the percentage of the budget for each sector should be given. The percentages should add up to 100. The same goes for recipient-country.

When you use both recipient-country and recipient-region, these percentages should add up to 100. For example:

Organisation B carries out a project that has an impact in Sub-Sahara Africa, but also aims at a specific impact in the Tanzania. Organisation B publishes:

<recipient-country code="TZ" percentage="40" />
<recipient-region code="289" vocabulary="1" percentage="60" />

Conclusion: Yeah! We have open data
I think we’ve made great steps forward: many organisations have published their first dataset(s), most are using identifiers to refer to each other and most datasets are fairly “rich” compared to the average IATI data.

As soon as we’ve visualized some of the data of course we’ll share it here.

I’m looking forward to the next round of publications. If in the meantime you have questions concerning your own IATI data, feel free to ask at or leave your question on this website.

IATI and traceability

One of the challenges with current IATI publications is the fact that relations between organisations and activities are not, or only partially defined. Therefore the ministry’s guiding principles for using IATI pay a much attention to clearly defining different kinds of relationships and the IATI elements that should be used to represent these relations.

A growing community of people is interested in this subject, given the discussions that take place at the IATI discussion forum. Rolf Kleef is also experimenting with traceability in IATI data.

Just as in real life, relations are what make IATI data interesting. So I was very pleased when Steven Flower reached out. He is working on traceability for UN-Habitat and mentioned that within the existing Minbuza and UN-Habitat IATI datasets, he’d found one example of a set of “paired activities” – activities that have a clear connection:

Looking at MinBuza’s activity description, the support from MinBuza indeed is earmarked for the MEKWATSAN programme.

Steven was wondering how this activity should be described following Minbuza’s guiding principles.

At the moment –looking at the IATI XML – there is only a weak link between the activities. Minbuza mentions UN-Habitat as the implementing organisation – with identifier 41120. UN-Habitat mentions ‘The Netherlands’ as the funding organisation, and 17 implementing organisations – with no identifiers. And that’s that.

So how do we get from the current situation to a more intelligent ‘family tree’?

Basically, three easy steps will do the trick:

  1. Use identifiers for the organisations you work with.
  2. Model the income side of your activities
  3. Show how your own activities are structured

From the available IATI data on both sides, this case seems pretty straightforward:

1 – Use identifiers

MinBuza already refers to UN-Habitat using the official identifier 41120.
UN-Habitat should refer to MinBuza using their organisation identifier (XM-DAC-7). And, where available, should also use identifiers when referring to its implementing partners.

2 – Model the income side of your activities

As a recipient UN-Habitat should also refer to the activity from which it is funded, in this case NL-1-PPR-18540)


3- Show how your activities are structured

Whether UN-Habitat presents this as a single activity or as an activity with more ‘children’ (e.g. a program with implementing projects) depends on how UN-Habitat manages the activity.

Looking at the description of the MinBuza activity I could imagine a subdivision into 8 implementing activities: “Supporting drink-water and sanitation programmes in eight secundary urban centres in the Mekong Region”. But one activity with outflows to 17 implementing organisations is also fine, if that’s an adequate model of what actually happens.

These three simple steps will already lead to a more complete picture of the activities at hand. As said: what that picture actually looks like depends on how UN-Habitat manages the activity.

Next: more family relations?

Things could get even more interesting when you look at the project document attached to UN-Habitat’s activity: this activity (the MEKWATSAN programme) appears to be part of a (very) large Water and Sanitation Programme/Trust Fund. It would be really interesting to see which donors contribute to that fund, and which programmes are defined to implement its strategic goals. It may not be the easiest exercise, but it would provide a lot of insight – and offer UN-Habitat the chance to show its portfolio and highlight the coherence of its activities.

I’m looking forward to see the next UN-Habitat data set. If you have more examples of networks built out of IATI data, please share them in the comments.