Utility functions

Supporting functions used by other modules.

APIs

block_until_ready(client, socket, env)

Block current thread until we receive the ready status :param client: the current connected client :type client: SSHClient :param socket: socket connection to machine where job is executed :param env: environment to set up on the remote machine

check_missing_key(config, key, msg, log=<Logger aup.utils (WARNING)>)

Verify important key is present, otherwise raise exception

Parameters

• config (dict) – configuration

• key (str) – key name

• msg (str) – warning message when key is not present

• log (logging.Logger) – logger obj to trace where the function is called, default is aup.utils

get_available_port()

Returns first open port available.

get_default_connector(auppath='.aup', log=<Logger aup.utils (WARNING)>)

Get default connector based on .env

Parameters

• auppath – aup environment folder, contains env.ini file

• log (logging.Logger) – logger obj to trace where the function is called, default is aup.utils

Returns

Database connector

Return type

aup.ET.Connector.AbstractConnector

get_default_username(username=None)

Get default username for Auptimizer

Parameters

username – username

Returns

username

get_from_options(msg, values, default=None)

helper function to ask from user input, and validate it is one of the permitted values.

Parameters

• msg – message to ask the user

• values – permitted values

• default – default value when user skip, if None, values[0] is used.

Returns

value with the same type as values[0]

load_default_env(auppath='.aup', log=<Logger aup.utils (WARNING)>, use_default=True)

Load default environment variables for aup. Search recursively to upper folder

Parameters

• auppath (str) – aup environment folder, contains env.ini file

• log (logging.Logger) – logger obj to trace where the function is called, default is aup.utils, if None, then no logging is performed

• use_default (bool) – if auppath is empty, use user’s home folder instead.

Returns

key-value of parameters

Return type

dict

open_sftp_with_timeout(client, timeout=5)

SFTP Connection to transfer files and environment between Auptimizer source machine and remote machine.

parse_one_line(result, log=<Logger aup.utils (WARNING)>)

parse_result(result, log=<Logger aup.utils (WARNING)>)

Parse the result printed by print_result()

Parameters

• log (logging.Logger) – logger obj to trace where the function is called, default is aup.utils

• result (str) – result string

Returns

result for Auptimizer update

Return type

float

set_default_keyvalue(key, value, d, inplace=True, log=<Logger aup.utils (WARNING)>)

Set default value for dict if key is not defined

Parameters

• key (str) – key name

• value (object) – key value

• d (dict) – target dict object to be updated

• inplace (boolean) – whether update in place.

• log (logging.Logger) – logger obj to trace where the function is called, default is aup.utils

Returns

updated dict

Return type

dict

Auptimizer client side functions

This file can be copied to a remote machine instead of installing the whole Auptimizer package for job execution.

APIs

class BasicConfig

Bases: dict

User-friendly dict supports:

• load and save for json/pickle format (.json/.pkl)

• easy key/value access as config.key or config[“key”]

• compatible with dict

Parameters

kwargs (dict) – key-value pairs to initialize the configuration

load(filename)

Load config parameters from JSON/pickle file

Parameters

filename (string) – file name ends with [.json|.pkl]

Returns

configuration parsed from file

Return type

aup.BasicConfig

save(filename)

Save configuration as dict in JSON/pickle

Parameters

filename (string) – file name ends with [.json|.pkl]

static save_flags(filename)

Save tf flags for reuse - not used, not tested

Parameters

filename – output file

to_flags(FLAGS)

Update values in FLAGS from BasicConfig

Parameters

FLAGS – tensorflow/absl FLAGS

aup_args(func)

Decorator to wrap optimization target function func.

Arguments:

func {function} – A function computes optimization target with specified hyperparameters

aup_flags(flags)

wrapper function for absl flags (or tf.app).

It will assign values to flags parameters using the given configuration file as the first argument when executed from the command line.

Arguments:

args {list} – a list of unused arguments passed by app.run()

aup_save_model(callback_fn, *args, **kwargs)

print_result(result)

Function to print the result for parse_result(). This function should be the last line of your training code

Parameters

result (str) – result from training code