cctools
|
Python PythonTask object. More...
Public Member Functions | |
__init__ (self, func, *args, **kwargs) | |
Creates a new python task. | |
output (self) | |
returns the result of a python task as a python variable | |
![]() | |
clone (self) | |
Return a copy of this task. | |
specify_command (self, command) | |
Set the command to be executed by the task. | |
specify_coprocess (self, coprocess) | |
Set the coprocess at the worker that should execute the task's command. | |
specify_algorithm (self, algorithm) | |
Set the worker selection algorithm for task. | |
specify_tag (self, tag) | |
Attach a user defined logical name to the task. | |
specify_category (self, name) | |
Label the task with the given category. | |
specify_feature (self, name) | |
Label the task with the given user-defined feature. | |
specify_file (self, local_name, remote_name=None, type=None, flags=None, cache=None, failure_only=None) | |
Add a file to the task. | |
specify_url (self, url, remote_name, type=None, flags=None, cache=None, failure_only=None) | |
Add a url to the task which will be provided as an input file. | |
specify_file_command (self, cmd, remote_name, type=None, flags=None, cache=None, failure_only=None) | |
Add an input file produced by a Unix shell command. | |
specify_file_piece (self, local_name, remote_name=None, start_byte=0, end_byte=0, type=None, flags=None, cache=None, failure_only=None) | |
Add a file piece to the task. | |
specify_input_file (self, local_name, remote_name=None, flags=None, cache=None) | |
Add a input file to the task. | |
specify_output_file (self, local_name, remote_name=None, flags=None, cache=None, failure_only=None) | |
Add a output file to the task. | |
specify_directory (self, local_name, remote_name=None, type=None, flags=None, recursive=False, cache=None, failure_only=None) | |
Add a directory to the task. | |
specify_buffer (self, buffer, remote_name, flags=None, cache=None) | |
Add an input bufer to the task. | |
specify_snapshot_file (self, filename) | |
When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources. | |
specify_max_retries (self, max_retries) | |
Indicate the number of times the task should be retried. | |
specify_cores (self, cores) | |
Indicate the number of cores required by this task. | |
specify_memory (self, memory) | |
Indicate the memory (in MB) required by this task. | |
specify_disk (self, disk) | |
Indicate the disk space (in MB) required by this task. | |
specify_gpus (self, gpus) | |
Indicate the number of GPUs required by this task. | |
specify_priority (self, priority) | |
Indicate the the priority of this task (larger means better priority, default is 0). | |
specify_environment_variable (self, name, value=None) | |
Set this environment variable before running the task. | |
specify_monitor_output (self, directory) | |
Set a name for the resource summary output directory from the monitor. | |
tag (self) | |
Get the user-defined logical name for the task. | |
category (self) | |
Get the category name for the task. | |
command (self) | |
Get the shell command executed by the task. | |
priority (self) | |
Get the priority of the task. | |
algorithm (self) | |
Get the algorithm for choosing worker to run the task. | |
std_output (self) | |
Get the standard output of the task. | |
id (self) | |
Get the task id number. | |
return_status (self) | |
Get the exit code of the command executed by the task. | |
result (self) | |
Get the result of the task as an integer code, such as successful, missing file, etc. | |
result_str (self) | |
Return a string that explains the result of a task. | |
total_submissions (self) | |
Get the number of times the task has been resubmitted internally. | |
exhausted_attempts (self) | |
Get the number of times the task has been failed given resource exhaustion. | |
host (self) | |
Get the address and port of the host on which the task ran. | |
hostname (self) | |
Get the name of the host on which the task ran. | |
submit_time (self) | |
Get the time at which this task was submitted. | |
finish_time (self) | |
Get the time at which this task was finished. | |
total_cmd_exhausted_execute_time (self) | |
Get the total time the task executed and failed given resource exhaustion. | |
app_delay (self) | |
Get the time spent in upper-level application (outside of work_queue_wait). | |
send_input_start (self) | |
Get the time at which the task started to transfer input files. | |
send_input_finish (self) | |
Get the time at which the task finished transferring input files. | |
execute_cmd_start (self) | |
The time at which the task began. | |
execute_cmd_finish (self) | |
Get the time at which the task finished (discovered by the manager). | |
receive_output_start (self) | |
Get the time at which the task started to transfer output files. | |
receive_output_finish (self) | |
Get the time at which the task finished transferring output files. | |
total_bytes_received (self) | |
Get the number of bytes received since task started receiving input data. | |
total_bytes_sent (self) | |
Get the number of bytes sent since task started sending input data. | |
total_bytes_transferred (self) | |
Get the number of bytes transferred since task started transferring input data. | |
total_transfer_time (self) | |
Get the time comsumed in microseconds for transferring total_bytes_transferred. | |
cmd_execution_time (self) | |
Time spent in microseconds for executing the command until completion on a single worker. | |
total_cmd_execution_time (self) | |
Accumulated time spent in microseconds for executing the command on any worker, regardless of whether the task finished (i.e., this includes time running on workers that disconnected). | |
resources_measured (self) | |
Get the resources measured for the task execution if resource monitoring is enabled. | |
limits_exceeded (self) | |
Get the resources the task exceeded. | |
resources_requested (self) | |
Get the resources the task requested to run. | |
resources_allocated (self) | |
Get the resources allocated to the task in its latest attempt. | |
Protected Member Functions | |
_create_wrapper (self) | |
creates the wrapper script which will execute the function. | |
Python PythonTask object.
this class is used to create a python task
ndcctools.work_queue.PythonTask.__init__ | ( | self, | |
func, | |||
* | args, | ||
** | kwargs | ||
) |
Creates a new python task.
self | Reference to the current python task object |
func | python function to be executed by task |
args | arguments used in function to be executed by task |
kwargs | keyword arguments used in function to be executed by task |
Reimplemented from ndcctools.work_queue.Task.
ndcctools.work_queue.PythonTask.output | ( | self | ) |
returns the result of a python task as a python variable
self | reference to the current python task object |
Reimplemented from ndcctools.work_queue.Task.
|
protected |
creates the wrapper script which will execute the function.
pickles output.