Fair use policy

This document details the fair use policy for the KNMI Data Platform (KDP).

All datasets on this platform are free of charge and are provided under the Wet Open Overheid (Woo) policy of the Dutch government. Providing them does cost money, and the capacity of the platform is shared by all of its users. This policy describes what we consider reasonable use, so that we can keep access open, free, and reliable for everyone.

One account per user

Register one account, with an email address that you can be reached on permanently. Do not create multiple accounts to obtain more quota or a higher rate limit.

Accounts and keys are personal. They are meant to be used by you or by your organisation for your own applications, and should not be shared with, resold to, or distributed to third parties.

Respect the rate limits and quotas

Every API key has a rate limit and an hourly quota. The applicable values are documented on the page of each API.

When you exceed a rate limit you receive an HTTP 429 response. Treat that as a signal to slow down:

  • Wait before you retry, and increase the waiting time on every following 429 (exponential backoff).
  • Never retry immediately in a tight loop. Repeatedly hammering an endpoint that is answering 429 is abuse, even though every individual request is rejected.
  • Handle HTTP 5xx responses the same way. If the platform has a problem, retrying harder makes it worse.
  • Spread scheduled jobs out over the hour instead of starting them all exactly on the hour or on the minute.

Do not poll excessively

Polling an API to find out whether something has changed is the most common cause of unnecessary load.

  • To learn about new files as soon as they are available, use the Notification Service. You receive a message for every new file, containing the Open Data API URL to download it. This is both faster and cheaper than polling.
  • If you cannot use the Notification Service, do not poll more often than the dataset is actually updated. Polling a dataset that is updated every 10 minutes once per second gains you nothing.
  • Request only what you need. Use the maxKeys, sorting, orderBy and begin parameters of the Open Data API to retrieve the files you are interested in, instead of listing an entire dataset to find the newest file.

Do not redownload a dataset

Do not download files you already have. Keep track of what you retrieved and store it on your own systems.

To download a complete dataset, request a dataset bulk key as described in the Open Data API documentation. Download a dataset once and keep it. Re-downloading a complete dataset because it is easier than archiving it locally is not fair use, and bulk keys are issued with a quota adequate for a single download.

Keep your API key secret

Your API key identifies you and consumes your quota. Anyone who has it can use up your limits, and to us their traffic looks like yours.

  • Never publish an API key in a public place: a public Git repository, a notebook, an issue tracker, a container image, a blog post, a support ticket, or documentation.
  • Never embed an API key in a public web client, mobile app, browser extension or WMS client configuration that you distribute to your users. Anything shipped to a browser or a device is public, even when it is minified or obfuscated.
  • If you want to serve KDP data to your own users, proxy the requests through your own backend, keep the key there, and add caching in your backend. The same advice applies to WMS layers in a web viewer and EDR API queries.
  • For public, unauthenticated clients you can use the anonymous key documented in the Open Data API documentation. Keep in mind that its limits are shared with all other anonymous users and that it expires yearly.
  • Store keys outside your source code, for example in an environment variable or a secret store.
  • If a key has leaked, contact us so we can revoke it. You can request a new key once the old one has been revoked.

Respect the data license

The data is free to use, also commercially, but not without conditions. Every dataset states its license in its metadata in the KNMI Data Platform Data Catalog. For most datasets this is CC BY 4.0, which allows you to use, adapt, and redistribute the data, as long as you credit KNMI as the source.

  • Check the license of the datasets you use before you publish anything based on them. Not all datasets carry the same license, and some originate from third parties that attach their own conditions to the data.
  • Credit KNMI as the source, for example with “Source: KNMI” and a link to the dataset in the Data Catalog.
  • Make clear when you show data that you derived or modified yourself, so that your users do not mistake it for an official KNMI product.
  • Do not suggest that KNMI endorses your application or that it is a KNMI product.

Be a recognisable client

  • Send a descriptive User-Agent header that identifies your application, and, if possible, how to reach you. It allows us to contact you about a problem instead of just blocking traffic.
  • For the Notification Service, use one stable, unique client ID per client, and do not open many parallel MQTT sessions. Reconnect with backoff, not in a loop.

Unacceptable use

Besides the above, we consider the following abuse of the KNMI Data Platform:

  • Circumventing rate limits, quotas, or expiry dates, for example by rotating accounts, keys, or IP addresses.
  • Distributed downloading of the same data from many clients.
  • Any attempt to disrupt the availability of the platform, or security testing of the platform without our written permission.

What happens if you exceed this policy

Our first step is to get in touch. If we see traffic that is harmful to the platform or to other users, we would much rather contact you and work it out together than simply block the traffic. Usually a short exchange is enough: we explain what we see, you adjust your client, and you keep your access.

Depending on the impact, we may in the meantime, and if needed without prior notice, revoke the API key, or delete the account.

Exceptions

No policy can cover every possible situation. KDP is an evolving platform and this document will evolve with it. If this policy does not fit your use case, or if you are unsure whether your usage is acceptable, contact us and describe your use case. We would much rather help you build something that works than block it later.

Navigation