dlhub_sdk package

Subpackages

• dlhub_sdk.models package
  • Subpackages
    • dlhub_sdk.models.servables package
      • Subpackages
      • Submodules
      • dlhub_sdk.models.servables.keras module
      • dlhub_sdk.models.servables.pytorch module
      • dlhub_sdk.models.servables.python module
      • dlhub_sdk.models.servables.sklearn module
      • dlhub_sdk.models.servables.tensorflow module
      • Module contents
  • Submodules
  • dlhub_sdk.models.datasets module
  • dlhub_sdk.models.pipeline module
  • Module contents
• dlhub_sdk.utils package
  • Submodules
  • dlhub_sdk.utils.schemas module
  • dlhub_sdk.utils.types module
  • dlhub_sdk.utils.search module
  • Module contents

Submodules

dlhub_sdk.client module

class dlhub_sdk.client.DLHubClient(dlh_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, search_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, fx_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, openid_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, sl_authorizer: Optional[globus_sdk.authorizers.base.GlobusAuthorizer] = None, http_timeout: Optional[int] = None, force_login: bool = False, **kwargs)

Bases: globus_sdk.client.BaseClient

Main class for interacting with the DLHub service

Holds helper operations for performing common tasks with the DLHub service. For example, get_servables produces a list of all servables registered with DLHub.

The initializer offers several routes for authenticating with the services:

1. Token-based authentication. The default route for login is to log in with your own

   Globus-associated account. Provided no arguments, the DLHub client will ask you to authenticate with Globus Auth then store your login tokens on your computer. The DLHub SDK will re-use these tokens unless you later call logout() or initialize the client with ``force_login=True`.

2. Pre-defined authorizers. The alternate route is to create authorizers using the Globus SDK directly

   and providing that authorizer to the initializer (e.g., DLHubClient(dlhub_authorizer=auth)). You must provide authorizers DLHub for all sub-services: Globus Search, FuncX, and OpenID.

clear_funcx_cache(servable=None)

Remove functions from the cache. Either remove a specific servable or wipe the whole cache.

Parameters:Servable – str The name of the servable to remove. Default None

describe_methods(name, method=None)

Get the description for the method(s) of a certain servable

Parameters:

• name (string) – DLHub name of the servable of the form <user>/<servable_name>
• method (string) – Optional: Name of the method

Returns:

Description of a certain method if method provided, all methods

if the method name was not provided.

Return type:

dict

describe_servable(name)

Get the description for a certain servable

Parameters:name (string) – DLHub name of the servable of the form <user>/<servable_name> Returns:Summary of the servable Return type:dict

easy_publish(title: str, creators: Union[str, List[str]], short_name: str, servable_type: str, serv_options: Dict[str, Any], affiliations: List[Sequence[str]] = None, paper_doi: str = None)

Simplified publishing method for servables

Parameters:

• title (string) – title for the servable
• creators (string | list) – either the creator’s name (FamilyName, GivenName) or a list of the creators’ names
• short_name (string) – shorthand name for the servable
• servable_type (string) – the type of the servable, must be a member of (“static_method”, “class_method”, “keras”, “pytorch”, “tensorflow”, “sklearn”) more information on servable types can be found here: https://dlhub-sdk.readthedocs.io/en/latest/servable-types.html
• serv_options (dict) – the servable_type specific arguments that are necessary for publishing
• affiliations (list) – list of affiliations for each author
• paper_doi (str) – DOI of a paper that describes the servable

Returns:

task id of this submission, can be used to check for success

Return type:

(string)

Raises:

• ValueError – If the given servable_type is not in the list of acceptable types
• Exception – If the serv_options are incomplete or the request to publish results in an error

get_result(task_id, verbose=False)

Get the result of a task_id

Parameters:

• str (task_id) – The task’s uuid
• bool (verbose) – whether or not to return the full dlhub response

Returns:

Reult of running the servable

get_servables(only_latest_version=True)

Get all of the servables available in the service

Parameters:only_latest_version (bool) – Whether to only return the latest version of each servable Returns:([list]) Complete metadata for all servables found in DLHub

get_task_status(task_id)

Get the status of a DLHub task.

Parameters:task_id (string) – UUID of the task Returns:status block containing “status” key. Return type:dict

get_username()

Get the username associated with the current credentials

list_servables()

Get a list of the servables available in the service

Returns:List of all servable names in username/servable_name format Return type:[string]

logout()

Remove credentials from your local system

publish_repository(repository)

Submit a repository to DLHub for publication

Parameters:repository (string) – Repository to publish Returns:Container ID of this submission, as returned from the Container Service Return type:(string)

publish_servable(model)

Submit a servable to DLHub

If this servable has not been published before, it will be assigned a unique identifier.

If it has been published before (DLHub detects if it has an identifier), then DLHub will update the servable to the new version.

Parameters:model (BaseMetadataModel) – Servable to be submitted Returns:Container ID of this submission, as returned from the Container Service Return type:(string)

query

Access a query of the DLHub Search repository

run(name: str, inputs: Any, parameters: Optional[Dict[str, Any]] = None, asynchronous: bool = False, debug: bool = False, validate_input: bool = False, async_wait: float = 5, timeout: Optional[float] = None) → Union[dlhub_sdk.utils.futures.DLHubFuture, Tuple[Any, Dict[str, Any]], Any]

Invoke a DLHub servable

Parameters:

• name – DLHub name of the servable of the form <user>/<servable_name>
• inputs – Data to be used as input to the function. Can be a string of file paths or URLs
• parameters – Any optional parameters to pass to the function.
• asynchronous – Whether to return from the function immediately or wait for the execution to finish.
• debug – Whether to capture the standard out and error printed during execution
• validate_input – whether to validate the provided input against the servable’s published metadata
• async_wait – How many seconds to wait between checking async status
• timeout – How long to wait for a result to return. Only used for synchronous calls

Returns:

If asynchronous, a DLHubFuture for the execution If debug, the output of the function and dictionary holding the following information:

• success: Whether the code inside ran without raising an exception
• stdout/stderr: Captured standard output and error, if requested
• timing: Execution time for the segment in seconds
• exc: Captured exception object
• error_message: Exception traceback

If neither, the output of the function

run_serial(servables, inputs, async_wait=5)

Invoke each servable in a serial pipeline. This function accepts a list of servables and will run each one, passing the output of one as the input to the next.

Parameters:

• servables (list) – A list of servable strings
• inputs – Data to pass to the first servable
• asycn_wait (float) – Seconds to wait between status checks

Returns:

Results of running the servable

search(query, advanced=False, limit=None, only_latest=True)

Query the DLHub servable library

By default, the query is used as a simple plaintext search of all model metadata. Optionally, you can provided an advanced query on any of the indexed fields in the DLHub model metadata by setting advanced=True and following the guide for constructing advanced queries found in the Globus Search documentation.

Parameters:

• query (string) – Query to be performed
• advanced (bool) – Whether to perform an advanced query
• limit (int) – Maximum number of entries to return
• only_latest (bool) – Whether to return only the latest version of the model

Returns:

All records matching the search query

Return type:

([dict])

search_by_authors(authors, match_all=True, limit=None, only_latest=True)

Execute a search for servables from certain authors.

Authors in DLHub may be different than the owners of the servable and generally are the people who developed functionality of a certain servable (e.g., the creators of the machine learning model used in a servable).

If you want to search by ownership, see search_by_servable()

Parameters:

• authors (str or list of str) – The authors to match. Names must be in “Family Name, Given Name” format
• match_all (bool) – If True, will require all authors be on any results. If False, will only require one author to be in results. Default: True.
• limit (int) – The maximum number of results to return. Default: None, for no limit.
• only_latest (bool) – When True, will only return the latest version of each servable. When False, will return all matching versions. Default: True.

Returns:

List of servables from the desired authors

Return type:

[dict]

search_by_related_doi(doi, limit=None, only_latest=True)

Get all of the servables associated with a certain publication

Parameters:

• doi (string) – DOI of related paper
• limit (int) – Maximum number of results to return
• only_latest (bool) – Whether to return only the most recent version of the model

Returns:

List of servables from the requested paper

Return type:

[dict]

search_by_servable(servable_name=None, owner=None, version=None, only_latest=True, limit=None, get_info=False)

Search by the ownership, name, or version of a servable

Parameters:

• servable_name (str) – The name of the servable. Default: None, to match all servable names.
• owner (str) – The name of the owner of the servable. Default: None, to match all owners.
• version (int) – Model version, which corresponds to the date when the servable was published. Default: None, to match all versions.
• only_latest (bool) – When True, will only return the latest version of each servable. When False, will return all matching versions. Default: True.
• limit (int) – The maximum number of results to return. Default: None, for no limit.
• get_info (bool) – If False, search will return a list of the results. If True, search will return a tuple containing the results list and other information about the query. Default: False.

Returns:

The search results. If info is True, tuple: The search results, and a dictionary of query information.

Return type:

If info is False, list

service_name = 'DLHub'

exception dlhub_sdk.client.HelpMessage

Bases: Exception

Raised from another error to provide the user an additional message

Module contents