The zarr utils command line interface#
This section introduces the usage of the freva-client zarr-utils sub command.
Please see the Installation and configuration section on how to install and
configure the command line interface.
After successful installation you can use the freva-client zarr-utils sub
command
freva-client zarr-utils --help
Converting your data files#
You can convert any data files on the data center to zarr stores and serve them
via http(s). To do so use the convert sub-command:
freva-client zarr-utils convert --help
Results
Usage: freva-client zarr-utils convert [OPTIONS] PATHS...
Convert different data sets (files) to a http ready zarr store.
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ * paths PATHS... Paths to data files that should be aggregated. │
│ [required] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --public Make any zarr url │
│ public │
│ --ttl-seconds FLOAT Set the expiry time of │
│ any public zarr urls │
│ [default: 86400.0] │
│ --aggregate [auto|merge|concat] How aggregation should │
│ be realised (if any) │
│ --join [outer|inner|exact|l How different indexes │
│ eft|right] should be combined for │
│ aggregation. │
│ --compat [no_conflicts|equals How to compare │
│ |override] non-concatenated │
│ variables for │
│ aggregation. │
│ --data-vars [minimal|different|a How to combine data │
│ ll] variables for │
│ aggregation. │
│ --coords [minimal|different|a How to combine coords │
│ ll] for aggregation. │
│ --dim TEXT Name of the dimension │
│ to concatenate along │
│ for aggregation. │
│ --group-by TEXT If set, forces │
│ grouping by a │
│ signature key for │
│ aggregation. │
│ --access-pattern [map|time_series] Optimise the chunk │
│ sizes for those access │
│ pattern. │
│ [default: map] │
│ --map-primary-chunks… INTEGER Chunk sizes of the │
│ primary dimension. │
│ [default: 1] │
│ --chunk-size FLOAT Set the target chunk │
│ size in megabytes. │
│ [default: 16.0] │
│ --reload-zarr Trigger a zarr │
│ data-store reload. │
│ --token-file -tf PATH Instead of │
│ authenticating via │
│ code based │
│ authentication flow │
│ you can set the path │
│ to the json file that │
│ contains a `refresh │
│ token` containing a │
│ refresh_token key. │
│ --json -j Parse output in json │
│ format. │
│ --host TEXT Set the hostname of │
│ the databrowser, if │
│ not set (default) the │
│ hostname is read from │
│ a config file │
│ -v INTEGER Increase verbosity │
│ [default: 0] │
│ --version -V Show version an exit │
│ --help Show this message and │
│ exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
The --public flag lets you create zarr stores that can be anonymously acceesed.
The aggregation flags can be used to aggregate multiple source file into one common zarr store.
Warning
Anyone with this link will be able to access the data. For security reasons any public zarr store link has an expiration date after which access isn’t possible anymore.
Checking the conversion status#
If client tools, like xarray, that access the zarr stores file you can use the
status sub-command to get more information about the loading process and
potential issues.
freva-client zarr-utils status --help
Results
Usage: freva-client zarr-utils status [OPTIONS] URL
Get the status of a pre signed zarr store
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ * url TEXT Url of the zarr store to check. [required] │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --token-file -tf PATH Instead of authenticating via code based │
│ authentication flow you can set the path to │
│ the json file that contains a `refresh │
│ token` containing a refresh_token key. │
│ --host TEXT Set the hostname of the databrowser, if not │
│ set (default) the hostname is read from a │
│ config file │
│ -v INTEGER Increase verbosity [default: 0] │
│ --version -V Show version an exit │
│ --help Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯