You can use the Nomisweb API to discover (list) datasets and to obtain them once you have identified what you need. The API supports a variety of formats but in this tutorial I use JSON because it’s less verbose than other formats so is a smaller file.
To list all the available datasets I used a programme called
jq (‘Json Query’ I assume).
You can install
jq on an Ubuntu machine with
sudo apt install jq or
sudo snap install jq.
You should install a version >= 1.5.
You will also need
sudo apt install curl).
The following script uses curl and jq to obtain and filter the results, retaining ID, name, and description.
It then saves the output to
- Line 1 specifies which shell to use.
- Line 2 specifies the relevant json file listing all the datasets, specified in the API documentation.
- Line 3 is a series of filters piped together. It essentially obtains everything; then filters out everything but
keyfamiliesat that level; then filters everything but
keyfamily(it just so happens there’s nothing to filter at that level); then obtains all datasets (’
.’); before finally extracting the id, name, and description from each index.
- Line 4 saves this to the file rather than print it to the terminal window as is the default.
If you’re an R user, querying and obtaining data from the Nomis API is now even easier with the