Project

class neptune.projects.Project(client, internal_id, namespace, name)[source]

Bases: object

A class for storing information and managing Neptune project.

Parameters
  • client (Client, required) – Client object.

  • internal_id (str, required) – UUID of the project.

  • namespace (str, required) – It can either be your organization or user name.

  • name (str, required) – project name.

Note

namespace and name joined together form project_qualified_name.

create_experiment(name=None, description=None, params=None, properties=None, tags=None, upload_source_files=None, abort_callback=None, logger=None, upload_stdout=True, upload_stderr=True, send_hardware_metrics=True, run_monitoring_thread=True, handle_uncaught_exceptions=True, git_info=None, hostname=None, notebook_id=None)[source]

Create and start Neptune experiment.

Create experiment, set its status to running and append it to the top of the experiments view. All parameters are optional, hence minimal invocation: neptune.create_experiment().

Parameters
  • name (str, optional, default is 'Untitled') – Editable name of the experiment. Name is displayed in the experiment’s Details (Metadata section) and in experiments view as a column.

  • description (str, optional, default is '') – Editable description of the experiment. Description is displayed in the experiment’s Details (Metadata section) and can be displayed in the experiments view as a column.

  • params (dict, optional, default is {}) – Parameters of the experiment. After experiment creation params are read-only (see: get_parameters()). Parameters are displayed in the experiment’s Details (Parameters section) and each key-value pair can be viewed in experiments view as a column.

  • properties (dict, optional, default is {}) – Properties of the experiment. They are editable after experiment is created. Properties are displayed in the experiment’s Details (Properties section) and each key-value pair can be viewed in experiments view as a column.

  • tags (list, optional, default is []) – Must be list of str. Tags of the experiment. They are editable after experiment is created (see: append_tag() and remove_tag()). Tags are displayed in the experiment’s Details (Metadata section) and can be viewed in experiments view as a column.

  • upload_source_files (list, optional, default is ['main.py']) –

    Where ‘main.py’ is Python file from which experiment was created (name ‘main.py’ is just an example here). Must be list of str. Uploaded sources are displayed in the experiment’s Source code tab.
    Pass empty list ([]) to upload no files.

  • abort_callback (callable, optional, default is None) –

    Callback that defines how abort experiment action in the Web application should work. Actual behavior depends on your setup:

    • (default) If abort_callback=None and psutil is installed, then current process and it’s children are aborted by sending SIGTERM. If, after grace period, processes are not terminated, SIGKILL is sent.

    • If abort_callback=None and psutil is not installed, then abort experiment action just marks experiment as aborted in the Web application. No action is performed on the current process.

    • If abort_callback=callable, then callable is executed when abort experiment action in the Web application is triggered.

  • logger (logging.handlers or None, optional, default is None) – If handler to Python logger is passed, new experiment’s text log (see: log_text()) with name “logger” is created. Each time Python logger logs new data, it is automatically sent to the “logger” in experiment. As a results all data from Python logger are in the Logs tab in the experiment.

  • upload_stdout (Boolean, optional, default is True) – Whether to send stdout to experiment’s Monitoring.

  • upload_stderr (Boolean, optional, default is True) – Whether to send stderr to experiment’s Monitoring.

  • send_hardware_metrics (Boolean, optional, default is True) – Whether to send hardware monitoring logs (CPU, GPU, Memory utilization) to experiment’s Monitoring.

  • run_monitoring_thread (Boolean, optional, default is True) – Whether to run thread that pings Neptune server in order to determine if experiment is responsive.

  • handle_uncaught_exceptions (Boolean, optional, default is True) –

    Two options True and False are possible:

    • If set to True and uncaught exception occurs, then Neptune automatically place Traceback in the experiment’s Details and change experiment status to Failed.

    • If set to False and uncaught exception occurs, then no action is performed in the Web application. As a consequence, experiment’s status is running or not responding.

  • git_info (GitInfo, optional, default is None) –

    Instance of the class GitInfo that provides information about the git repository from which experiment was started.
    If None is passed, system attempts to automatically extract information about git repository in the following way:
    • System looks for .git file in the current directory and, if not found, goes up recursively until .git file will be found (see: get_git_info()).

    • If there is no git repository, then no information about git is displayed in experiment details in Neptune web application.

  • hostname (str, optional, default is None) – If None, neptune automatically get hostname information. User can also set hostname directly by passing str.

Returns

Experiment object that is used to manage experiment and log data to it.

Raises
  • ExperimentValidationError – When provided arguments are invalid.

  • ExperimentLimitReached – When experiment limit in the project has been reached.

Examples

# minimal invoke
neptune.create_experiment()

# explicitly return experiment object
experiment = neptune.create_experiment()

# create experiment with name and two parameters
neptune.create_experiment(name='first-pytorch-ever',
                          params={'lr': 0.0005,
                                  'dropout': 0.2})

# create experiment with name and description, and no sources files uploaded
neptune.create_experiment(name='neural-net-mnist',
                          description='neural net trained on MNIST',
                          upload_source_files=[])

# larger example
neptune.create_experiment(name='first-pytorch-ever',
                          params={'lr': 0.0005,
                                  'dropout': 0.2},
                          properties={'key1': 'value1',
                                      'key2': 17,
                                      'key3': 'other-value'},
                          description='write longer description here',
                          tags=['list-of', 'tags', 'goes-here', 'as-list-of-strings'],
                          upload_source_files=['training_with_pytorch.py', 'net.py'])
create_notebook()[source]

Create a new notebook object and return corresponding Notebook instance.

Returns

Notebook object.

Examples

# Instantiate a session and fetch a project
project = neptune.init()

# Create a notebook in Neptune
notebook = project.create_notebook()
full_id

Project qualified name as str, for example john/sandbox.

get_experiments(id=None, state=None, owner=None, tag=None, min_running_time=None)[source]

Retrieve list of experiments matching the specified criteria.

All parameters are optional, each of them specifies a single criterion. Only experiments matching all of the criteria will be returned.

Parameters
  • id (str or list of str, optional, default is None) –

    An experiment id like 'SAN-1' or list of ids like ['SAN-1', 'SAN-2'].
    Matching any element of the list is sufficient to pass criterion.

  • state (str or list of str, optional, default is None) –

    An experiment state like 'succeeded' or list of states like ['succeeded', 'running'].
    Possible values: 'running', 'succeeded', 'failed', 'aborted'.
    Matching any element of the list is sufficient to pass criterion.

  • owner (str or list of str, optional, default is None) –

    Username of the experiment owner (User who created experiment is an owner) like 'josh' or list of owners like ['frederic', 'josh'].
    Matching any element of the list is sufficient to pass criterion.

  • tag (str or list of str, optional, default is None) –

    An experiment tag like 'lightGBM' or list of tags like ['pytorch', 'cycleLR'].
    Only experiments that have all specified tags will match this criterion.

  • min_running_time (int, optional, default is None) – Minimum running time of an experiment in seconds, like 2000.

Returns

list of Experiment objects.

Examples

# Fetch a project
project = session.get_projects('neptune-ml')['neptune-ml/Salt-Detection']

# Get list of experiments
project.get_experiments(state=['aborted'], owner=['neyo'], min_running_time=100000)

# Example output:
# [Experiment(SAL-1609),
#  Experiment(SAL-1765),
#  Experiment(SAL-1941),
#  Experiment(SAL-1960),
#  Experiment(SAL-2025)]
get_leaderboard(id=None, state=None, owner=None, tag=None, min_running_time=None)[source]

Fetch Neptune experiments view as pandas DataFrame.

returned DataFrame

In the returned DataFrame each row is an experiment and columns represent all system properties, numeric and text logs, parameters and properties in these experiments.
Note that, returned DataFrame does not contain all columns across the entire project.
Some columns may be empty, since experiments may define various logs, properties, etc.
For each log at most one (the last one) value is returned per experiment.
Text values are trimmed to 255 characters.

about parameters

All parameters are optional, each of them specifies a single criterion. Only experiments matching all of the criteria will be returned.

Parameters
  • id (str or list of str, optional, default is None) –

    An experiment id like 'SAN-1' or list of ids like ['SAN-1', 'SAN-2'].
    Matching any element of the list is sufficient to pass criterion.

  • state (str or list of str, optional, default is None) –

    An experiment state like 'succeeded' or list of states like ['succeeded', 'running'].
    Possible values: 'running', 'succeeded', 'failed', 'aborted'.
    Matching any element of the list is sufficient to pass criterion.

  • owner (str or list of str, optional, default is None) –

    Username of the experiment owner (User who created experiment is an owner) like 'josh' or list of owners like ['frederic', 'josh'].
    Matching any element of the list is sufficient to pass criterion.

  • tag (str or list of str, optional, default is None) –

    An experiment tag like 'lightGBM' or list of tags like ['pytorch', 'cycleLR'].
    Only experiments that have all specified tags will match this criterion.

  • min_running_time (int, optional, default is None) – Minimum running time of an experiment in seconds, like 2000.

Returns

pandas.DataFrame - Fetched Neptune experiments view.

Examples

# Fetch a project.
project = session.get_projects('neptune-ml')['neptune-ml/Salt-Detection']

# Get DataFrame that resembles experiment view.
project.get_leaderboard(state=['aborted'], owner=['neyo'], min_running_time=100000)
get_members()[source]

Retrieve a list of project members.

Returns

list of str - A list of usernames of project members.

Examples

project = session.get_projects('neptune-ml')['neptune-ml/Salt-Detection']
project.get_members()
get_notebook(notebook_id)[source]

Get a Notebook object with given notebook_id.

Returns

Notebook object.

Examples

# Instantiate a session and fetch a project
project = neptune.init()

# Get a notebook object
notebook = project.get_notebook('d1c1b494-0620-4e54-93d5-29f4e848a51a')