aup.ET.Connector package¶

aup.ET.Connector.AbstractConnector¶

Define the basic interface between experiment tracking and executor engine.

Currently, SQLite is the only one implemented.

APIs¶

class AbstractConnector[source]¶

Bases: abc.ABC

abstract close()[source]¶: Mark the connector as closed. A closed connector should not be used anymore.

create_experiment(username, name, exp_config_blob)[source]¶: Interface for creating an experiment without starting it

delete_experiment(eid)[source]¶: Interface for deleting an experiment by eid

abstract end_experiment(eid)[source]¶

Mark experiment as finished

Parameters: eid (int) – Experiment ID

abstract end_job(jid, score, status)[source]¶

Mark a job ended

Parameters

jid (int) – Job ID
score (str) – score of the Job
status (enumeration) – status of the Job

abstract end_job_attempt(jid)[source]¶

Ends a job attempt, but not a job (leaving room for retries)

Parameters: id (int) – Job ID

abstract free_used_resource(rid)[source]¶

Mark resource as free (opposite to :func:take_available_resource)

Parameters: rid (int) – Resource ID(s)

abstract get_all_experiment(username=None)[source]¶

Get all Experiment IDs

Parameters: username (str) – to get experiments for a specific user
Returns: Experiment IDs
Return type: list(int)

abstract get_all_history(eid)[source]¶

Get full history of an Experiment

Parameters: eid (int) – Experiment ID
Returns

abstract get_available_resource(username, rtype)[source]¶

Get available resource for a user and resource type. Currently there is no limitation/requirement for user

Parameters

username (str) – username
rtype (str) – Resource type

Returns

Resource Id

Return type

list(int)

abstract get_best_result(eid, maximize=True)[source]¶

Retrieve the best job id and score from the database for experiment eid

Parameters

eid (int) – Experiment ID
maximize (bool) – whether to choose max or min

Returns

Job ID, score

Return type

list(int, float)

get_experiment_status(eid)[source]¶: Interface for getting the database status of an experiment

get_intermediate_results_experiment(eid, status)[source]¶

Interface for extracting all intermediate results for all jobs of a given experiment, fitting a given status.

Parameters

eid (int) – Experiment ID
status (enumeration) – Desired status for filtering jobs

Returns

dictionary mapping job IDs to lists of intermediate results, in order

Return type

dict<int, float list>

get_intermediate_results_job(jid)[source]¶

Interface for extracting all intermediate results available for a given job.

Parameters: jid (int) – Job ID
Returns: list of intermediate results, in order
Return type: float list

get_intermediate_results_jobs(jids)[source]¶

Interface for extracting all intermediate results available for a given list of jobs.

Parameters: jids (int list) – Job IDs
Returns: dictionary mapping job IDs to lists of intermediate results, in order
Return type: dict<int, float list>

abstract get_resource_type()[source]¶: Get the resource type for a given user :return: list of resource types :rtype: list

abstract get_running_job(eid)[source]¶

Get running Job IDs

Parameters: eid (int) – Experiment ID
Returns: list of Job IDs
Return type: list(int)

abstract is_closed()[source]¶: Return whether or not the connector is closed. A closed connector should not be used anymore.

job_failed(rid, jid)[source]¶

Interface to take care of job failure in case of possible retries

Parameters

rid (int) – Resource ID
jid (int) – Job ID

job_finished(rid, jid, score=None, status=None)[source]¶

Clean up Job when it is finished

Parameters

rid (int) – Resource ID to be free
jid (int) – Job ID
score (float / str) – score returned by Job (error case will be ERROR)
status (enumeration) – job status

job_retry(rid, jid)[source]¶

Interface to mark the beginning of a job retry

Parameters

rid (int) – Resource ID
jid (int) – Job ID

job_started(eid, rid, job_config)[source]¶

Interface to automatically take resource and run job.

Parameters

eid (int) – Experiment ID
rid (int) – Resource ID
job_config (BasicConfig) – Configuration for Job

Returns

Job ID

Return type

int

log_error_message(eid, msg)[source]¶: Log in database the error message

save_intermediate_result(jid, score)[source]¶

Interface for saving intermediate result for a job.

Parameters

jid (int) – Job ID
score (float) – the intermediate result

Returns

intermediate_result table id

Return type

int

save_multiple_results(jid, irid, eid, labels, scores)[source]¶: Save in database other results desired by user

set_last_multiple_results(eid, jid, num_labels)[source]¶: Set the ‘is_last_result’ flag to true for this jid

abstract start_experiment(username, name, exp_config_blob)[source]¶

Create an Experiment and track it

Parameters

username (str) – username
name (str) – experiment name
exp_config_blob (str) – configuration of experiment

Returns

experiment ID

Return type

int

start_experiment_by_eid(eid)[source]¶: Interface for (re)starting an experiment by eid

abstract start_job(eid, rid, job_config)[source]¶

Start a job with job configuration and track it

Parameters

eid (int) – Experiment ID
rid (int) – Resource ID
job_config (BasicConfig) – Configuration for aup.EE.Job.Job

Returns

Job ID (jid)

Return type

int

abstract start_job_attempt(rid, jid)[source]¶

Starts a job attempt for a given job, using the given resource

Parameters

rid (int) – Resource ID
jid (int) – Job ID

Returns

Job Attempt ID (jaid)

Return type

int

abstract take_available_resource(rid)[source]¶

Mark resource as used

TODO - currently where is prevention for async update, might not be relevant in the near future

Parameters: rid (int) – Resource ID(s)
Returns: True/False

update_job_status(jid, status)[source]¶

Change a job’s status

Parameters

jid (int) – Job ID
status (enumeration) – new status

aup.ET.Connector.SQLiteConnector¶

If encounter “Failed to query SQLite after xx times” error, increase DELAY_INTERVAL and REPEATED_TIME to prevent problem temporarily.

APIs¶

class SQLiteConnector(filename)[source]¶

Bases: aup.ET.Connector.AbstractConnector.AbstractConnector

change_experiment_status(**kwargs)[source]¶

close(**kwargs)[source]¶: Mark the connector as closed. A closed connector should not be used anymore.

create_experiment(**kwargs)[source]¶: Interface for creating an experiment without starting it

delete_experiment(**kwargs)[source]¶: Interface for deleting an experiment by eid

end_experiment(**kwargs)[source]¶

Mark experiment as finished

Parameters: eid (int) – Experiment ID

end_job(**kwargs)[source]¶

Mark a job ended

Parameters

jid (int) – Job ID
score (str) – score of the Job
status (enumeration) – status of the Job

end_job_attempt(**kwargs)[source]¶

Ends a job attempt, but not a job (leaving room for retries)

Parameters: id (int) – Job ID

free_all_resources(**kwargs)[source]¶

free_used_resource(**kwargs)[source]¶

Mark resource as free (opposite to :func:take_available_resource)

Parameters: rid (int) – Resource ID(s)

get_all_experiment(**kwargs)[source]¶

Get all Experiment IDs

Parameters: username (str) – to get experiments for a specific user
Returns: Experiment IDs
Return type: list(int)

get_all_history(**kwargs)[source]¶

Get full history of an Experiment

Parameters: eid (int) – Experiment ID
Returns

get_available_resource(**kwargs)[source]¶

Get available resource for a user and resource type. Currently there is no limitation/requirement for user

Parameters

username (str) – username
rtype (str) – Resource type

Returns

Resource Id

Return type

list(int)

get_best_result(**kwargs)[source]¶

Retrieve the best job id and score from the database for experiment eid

Parameters

eid (int) – Experiment ID
maximize (bool) – whether to choose max or min

Returns

Job ID, score

Return type

list(int, float)

get_best_result_config(**kwargs)[source]¶

get_experiment_status(**kwargs)[source]¶: Interface for getting the database status of an experiment

get_intermediate_results_experiment(**kwargs)[source]¶

Interface for extracting all intermediate results for all jobs of a given experiment, fitting a given status.

Parameters

eid (int) – Experiment ID
status (enumeration) – Desired status for filtering jobs

Returns

dictionary mapping job IDs to lists of intermediate results, in order

Return type

dict<int, float list>

get_intermediate_results_job(**kwargs)[source]¶

Interface for extracting all intermediate results available for a given job.

Parameters: jid (int) – Job ID
Returns: list of intermediate results, in order
Return type: float list

get_intermediate_results_jobs(**kwargs)[source]¶

Interface for extracting all intermediate results available for a given list of jobs.

Parameters: jids (int list) – Job IDs
Returns: dictionary mapping job IDs to lists of intermediate results, in order
Return type: dict<int, float list>

get_resource_type(**kwargs)[source]¶: Get the resource type for a given user :return: list of resource types :rtype: list

get_running_job(**kwargs)[source]¶

Get running Job IDs

Parameters: eid (int) – Experiment ID
Returns: list of Job IDs
Return type: list(int)

is_closed(**kwargs)[source]¶: Return whether or not the connector is closed. A closed connector should not be used anymore.

log_error_message(**kwargs)[source]¶: Log in database the error message

maybe_get_experiment_status(**kwargs)[source]¶

save_intermediate_result(**kwargs)[source]¶

Interface for saving intermediate result for a job.

Parameters

jid (int) – Job ID
score (float) – the intermediate result

Returns

intermediate_result table id

Return type

int

save_multiple_results(**kwargs)[source]¶: Save in database other results desired by user

set_last_multiple_results(**kwargs)[source]¶: Set the ‘is_last_result’ flag to true for this jid

start_experiment(**kwargs)[source]¶

Create an Experiment and track it

Parameters

username (str) – username
name (str) – experiment name
exp_config_blob (str) – configuration of experiment

Returns

experiment ID

Return type

int

start_experiment_by_eid(**kwargs)[source]¶: Interface for (re)starting an experiment by eid

start_job(**kwargs)[source]¶

Start a job with job configuration and track it

Parameters

eid (int) – Experiment ID
rid (int) – Resource ID
job_config (BasicConfig) – Configuration for aup.EE.Job.Job

Returns

Job ID (jid)

Return type

int

start_job_attempt(**kwargs)[source]¶

Starts a job attempt for a given job, using the given resource

Parameters

rid (int) – Resource ID
jid (int) – Job ID

Returns

Job Attempt ID (jaid)

Return type

int

take_available_resource(**kwargs)[source]¶

Mark resource as used

TODO - currently where is prevention for async update, might not be relevant in the near future

Parameters: rid (int) – Resource ID(s)
Returns: True/False

update_job_status(**kwargs)[source]¶

Change a job’s status

Parameters

jid (int) – Job ID
status (enumeration) – new status