FAIR data pipeline manual

This is the manual for the upcoming FAIR DataPipeline package.

FAIR data pipeline manual

Managing code runs
Reading data
Writing data
Raising issues
Index

Managing code runs

DataPipeline.initialise — Function

initialise(config_file, submission_script)

Reads in working config.yaml file, generates a new Code Run entry, and returns a DataRegistryHandle containing various metadata.

source

DataPipeline.finalise — Function

finalise(handle)

Complete (i.e. finish) code run.

source

Reading data

DataPipeline.read_array — Function

read_array(handle, data_product[, component])

Read [array] data product.

note that it must already have been downloaded from the remote data store using fdp pull.
the latest version of the data is read unless otherwise specified.

source

DataPipeline.read_table — Function

read_table(handle, data_product[, component])

Read [table] data product.

note that it must already have been downloaded from the remote data store using fdp pull.
the latest version of the data is read unless otherwise specified.

source

DataPipeline.read_estimate — Function

read_estimate(handle, data_product, [component])

Read TOML-based data product.

note that it must already have been downloaded from the remote data store using fdp pull.
the specific version can be specified in the config file (else the latest version is used.)

source

DataPipeline.read_distribution — Function

readdistribution(handle, dataproduct, [component])

Read TOML-based data product.

note that it must already have been downloaded from the remote data store using fdp pull.
the specific version can be specified in the config file (else the latest version is used.)

source

Missing docstring.

Missing docstring for link_read. Check Documenter's build log for details.

Writing data

DataPipeline.write_array — Function

write_array(handle, data, data_product, component)

Write an array as a component to an hdf5 file.

source

DataPipeline.write_table — Function

write_table(handle, data, data_product, component)

Write a table as a component to an hdf5 file.

source

DataPipeline.write_estimate — Function

write_estimate(handle, value, data_product, component)

Write a point estimate as a component to a toml file.

source

DataPipeline.write_distribution — Function

write_distribution(handle, distribution, parameters, data_product, component)

Write a distribution as a component to a toml file.

source

Missing docstring.

Missing docstring for link_write. Check Documenter's build log for details.

Raising issues

DataPipeline.raise_issue — Function

raise_issue(handle; ... )

Pass the object URI as a named parameter[s], e.g. raise_issue(handle; data_product=dp, component=comp).

Optional parameters

data_product
component
external_object
script

source