dlhub_sdk package

Submodules

dlhub_sdk.client module

class dlhub_sdk.client.DLHubClient(dlh_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, search_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, fx_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, openid_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, http_timeout: Optional[int] = None, force_login: bool = False, **kwargs)

Bases: globus_sdk.client.BaseClient

Main class for interacting with the DLHub service

Holds helper operations for performing common tasks with the DLHub service. For example, get_servables produces a list of all servables registered with DLHub.

The initializer offers several routes for authenticating with the services:

  1. Token-based authentication. The default route for login is to log in with your own

    Globus-associated account. Provided no arguments, the DLHub client will ask you to authenticate with Globus Auth then store your login tokens on your computer. The DLHub SDK will re-use these tokens unless you later call logout() or initialize the client with ``force_login=True`.

  2. Pre-defined authorizers. The alternate route is to create authorizers using the Globus SDK directly

    and providing that authorizer to the initializer (e.g., DLHubClient(dlhub_authorizer=auth)). You must provide authorizers DLHub for all sub-services: Globus Search, FuncX, and OpenID.

clear_funcx_cache(servable=None)

Remove functions from the cache. Either remove a specific servable or wipe the whole cache.

Parameters:Servable – str The name of the servable to remove. Default None
describe_methods(name, method=None)

Get the description for the method(s) of a certain servable

Parameters:
  • name (string) – DLHub name of the servable of the form <user>/<servable_name>
  • method (string) – Optional: Name of the method
Returns:

Description of a certain method if method provided, all methods

if the method name was not provided.

Return type:

dict

describe_servable(name)

Get the description for a certain servable

Parameters:name (string) – DLHub name of the servable of the form <user>/<servable_name>
Returns:Summary of the servable
Return type:dict
easy_publish(title: str, creators: Union[str, List[str]], short_name: str, servable_type: str, serv_options: Dict[str, Any], affiliations: List[Sequence[str]] = None, paper_doi: str = None)

Simplified publishing method for servables

Parameters:
  • title (string) – title for the servable
  • creators (string | list) – either the creator’s name (FamilyName, GivenName) or a list of the creators’ names
  • short_name (string) – shorthand name for the servable
  • servable_type (string) – the type of the servable, must be a member of (“static_method”, “class_method”, “keras”, “pytorch”, “tensorflow”, “sklearn”) more information on servable types can be found here: https://dlhub-sdk.readthedocs.io/en/latest/servable-types.html
  • serv_options (dict) – the servable_type specific arguments that are necessary for publishing
  • affiliations (list) – list of affiliations for each author
  • paper_doi (str) – DOI of a paper that describes the servable
Returns:

task id of this submission, can be used to check for success

Return type:

(string)

Raises:
  • ValueError – If the given servable_type is not in the list of acceptable types
  • Exception – If the serv_options are incomplete or the request to publish results in an error
get_result(task_id, verbose=False)

Get the result of a task_id

Parameters:
  • str (task_id) – The task’s uuid
  • bool (verbose) – whether or not to return the full dlhub response
Returns:

Reult of running the servable

get_servables(only_latest_version=True)

Get all of the servables available in the service

Parameters:only_latest_version (bool) – Whether to only return the latest version of each servable
Returns:([list]) Complete metadata for all servables found in DLHub
get_task_status(task_id)

Get the status of a DLHub task.

Parameters:task_id (string) – UUID of the task
Returns:status block containing “status” key.
Return type:dict
get_username()

Get the username associated with the current credentials

list_servables()

Get a list of the servables available in the service

Returns:List of all servable names in username/servable_name format
Return type:[string]
logout()

Remove credentials from your local system

publish_repository(repository)

Submit a repository to DLHub for publication

Parameters:repository (string) – Repository to publish
Returns:Task ID of this submission, used for checking for success
Return type:(string)
publish_servable(model)

Submit a servable to DLHub

If this servable has not been published before, it will be assigned a unique identifier.

If it has been published before (DLHub detects if it has an identifier), then DLHub will update the servable to the new version.

Parameters:model (BaseMetadataModel) – Servable to be submitted
Returns:Task ID of this submission, used for checking for success
Return type:(string)
query

Access a query of the DLHub Search repository

run(name: str, inputs: Any, parameters: Optional[Dict[str, Any]] = None, asynchronous: bool = False, debug: bool = False, validate_input: bool = False, async_wait: float = 5, timeout: Optional[float] = None) → Union[dlhub_sdk.utils.futures.DLHubFuture, Tuple[Any, Dict[str, Any]], Any]

Invoke a DLHub servable

Parameters:
  • name – DLHub name of the servable of the form <user>/<servable_name>
  • inputs – Data to be used as input to the function. Can be a string of file paths or URLs
  • parameters – Any optional parameters to pass to the function.
  • asynchronous – Whether to return from the function immediately or wait for the execution to finish.
  • debug – Whether to capture the standard out and error printed during execution
  • validate_input – whether to validate the provided input against the servable’s published metadata
  • async_wait – How many seconds to wait between checking async status
  • timeout – How long to wait for a result to return. Only used for synchronous calls
Returns:

If asynchronous, a DLHubFuture for the execution If debug, the output of the function and dictionary holding the following information:

  • success: Whether the code inside ran without raising an exception
  • stdout/stderr: Captured standard output and error, if requested
  • timing: Execution time for the segment in seconds
  • exc: Captured exception object
  • error_message: Exception traceback

If neither, the output of the function

run_serial(servables, inputs, async_wait=5)

Invoke each servable in a serial pipeline. This function accepts a list of servables and will run each one, passing the output of one as the input to the next.

Parameters:
  • servables (list) – A list of servable strings
  • inputs – Data to pass to the first servable
  • asycn_wait (float) – Seconds to wait between status checks
Returns:

Results of running the servable

search(query, advanced=False, limit=None, only_latest=True)

Query the DLHub servable library

By default, the query is used as a simple plaintext search of all model metadata. Optionally, you can provided an advanced query on any of the indexed fields in the DLHub model metadata by setting advanced=True and following the guide for constructing advanced queries found in the Globus Search documentation.

Parameters:
  • query (string) – Query to be performed
  • advanced (bool) – Whether to perform an advanced query
  • limit (int) – Maximum number of entries to return
  • only_latest (bool) – Whether to return only the latest version of the model
Returns:

All records matching the search query

Return type:

([dict])

search_by_authors(authors, match_all=True, limit=None, only_latest=True)

Execute a search for servables from certain authors.

Authors in DLHub may be different than the owners of the servable and generally are the people who developed functionality of a certain servable (e.g., the creators of the machine learning model used in a servable).

If you want to search by ownership, see search_by_servable()

Parameters:
  • authors (str or list of str) – The authors to match. Names must be in “Family Name, Given Name” format
  • match_all (bool) – If True, will require all authors be on any results. If False, will only require one author to be in results. Default: True.
  • limit (int) – The maximum number of results to return. Default: None, for no limit.
  • only_latest (bool) – When True, will only return the latest version of each servable. When False, will return all matching versions. Default: True.
Returns:

List of servables from the desired authors

Return type:

[dict]

Get all of the servables associated with a certain publication

Parameters:
  • doi (string) – DOI of related paper
  • limit (int) – Maximum number of results to return
  • only_latest (bool) – Whether to return only the most recent version of the model
Returns:

List of servables from the requested paper

Return type:

[dict]

search_by_servable(servable_name=None, owner=None, version=None, only_latest=True, limit=None, get_info=False)

Search by the ownership, name, or version of a servable

Parameters:
  • servable_name (str) – The name of the servable. Default: None, to match all servable names.
  • owner (str) – The name of the owner of the servable. Default: None, to match all owners.
  • version (int) – Model version, which corresponds to the date when the servable was published. Default: None, to match all versions.
  • only_latest (bool) – When True, will only return the latest version of each servable. When False, will return all matching versions. Default: True.
  • limit (int) – The maximum number of results to return. Default: None, for no limit.
  • get_info (bool) – If False, search will return a list of the results. If True, search will return a tuple containing the results list and other information about the query. Default: False.
Returns:

The search results. If info is True, tuple: The search results, and a dictionary of query information.

Return type:

If info is False, list

service_name = 'DLHub'
exception dlhub_sdk.client.HelpMessage

Bases: Exception

Raised from another error to provide the user an additional message

Module contents