API

class materializationengine.blueprints.materialize.api.ActiveTasksResource(api=None, *args, **kwargs)[source]

Bases: Resource

get()[source]

Get running tasks from celery

methods: Optional[List[str]] = {'GET'}

A list of methods this view can handle.

class materializationengine.blueprints.materialize.api.AnnotationResource(api=None, *args, **kwargs)[source]

Bases: Resource

get(aligned_volume_name: str, version: int, tablename: str)[source]
methods: Optional[List[str]] = {'GET'}

A list of methods this view can handle.

class materializationengine.blueprints.materialize.api.BulkUploadResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str, table_name: str, segmentation_source: str, description: str)[source]

Run bulk upload from npy files

Parameters:
  • column_mappings (dict) – dict mapping file names to column names in database

  • project (str) – bucket project path

  • file_path (str) – bucket project path

  • schema (str) – type of schema from emannotationschemas

  • datastack_name (str) – name of datastack from infoservice

  • table_name (str) – name of table in database to create

  • segmentation_source (str) – source of segmentation data

  • description (str) – text field added to annotation metadata table for reference

class materializationengine.blueprints.materialize.api.CeleryResource(api=None, *args, **kwargs)[source]

Bases: Resource

get()[source]

Get celery worker info

methods: Optional[List[str]] = {'GET'}

A list of methods this view can handle.

class materializationengine.blueprints.materialize.api.CompleteWorkflowResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str)[source]

Create versioned materialization, finds missing segmentations and updates roots

Parameters:

datastack_name (str) – name of datastack from infoservice

class materializationengine.blueprints.materialize.api.CreateFrozenMaterializationResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str)[source]

Create a new frozen (versioned) materialization

Parameters:

datastack_name (str) – name of datastack from infoservice

class materializationengine.blueprints.materialize.api.CreateVirtualPublicVersionResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post()[source]

Create a virtual version from an existing frozen version.

Parameters:
  • datastack_name (str) – name of datastack

  • version (int) – version to make virtual copy

class materializationengine.blueprints.materialize.api.DatasetResource(api=None, *args, **kwargs)[source]

Bases: Resource

get(aligned_volume_name: str)[source]
methods: Optional[List[str]] = {'GET'}

A list of methods this view can handle.

class materializationengine.blueprints.materialize.api.InsertMissingChunks(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str, table_name: str, segmentation_source: str, description: str)[source]

Insert missing chunks of data into database

Parameters:
  • chunks (list) – list mapping file names to column names in database

  • datastack_name (str) – name of datastack from infoservice

  • table_name (str) – name of table in database to create

  • segmentation_source (str) – source of segmentation data

  • description (str) – text field added to annotation metadata table for reference

class materializationengine.blueprints.materialize.api.LockedTasksResource(api=None, *args, **kwargs)[source]

Bases: Resource

get()[source]

Get locked tasks from redis

methods: Optional[List[str]] = {'GET', 'PUT'}

A list of methods this view can handle.

put()[source]

Unlock locked tasks

class materializationengine.blueprints.materialize.api.LookupDenseMissingRootIdsResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str)[source]

Run workflow to lookup missing root ids and insert into database across all tables in the database.

Parameters:

datastack_name (str) – name of datastack from infoservice

class materializationengine.blueprints.materialize.api.LookupSparseMissingRootIdsResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str, table_name: str)[source]

Finds null root ids in a given table and lookups new root ids using last updated time stamp.

Parameters:
  • datastack_name (str) – name of datastack from infoservice

  • table_name (str) – name of table

class materializationengine.blueprints.materialize.api.ProcessNewAnnotationsResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str)[source]

Process newly added annotations and lookup segmentation data

Parameters:

datastack_name (str) – name of datastack from infoservice

class materializationengine.blueprints.materialize.api.ProcessNewAnnotationsTableResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str, table_name: str)[source]

Process newly added annotations and lookup segmentation data

Parameters:
  • datastack_name (str) – name of datastack from infoservice

  • table_name (str) – name of table

class materializationengine.blueprints.materialize.api.ProcessNewSVIDResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str, table_name: str)[source]

Process newly added annotations and lookup supervoxel data

Parameters:
  • datastack_name (str) – name of datastack from infoservice

  • table_name (str) – name of table

class materializationengine.blueprints.materialize.api.QueueResource(api=None, *args, **kwargs)[source]

Bases: Resource

get()[source]

Get queued tasks for celery workers

methods: Optional[List[str]] = {'GET'}

A list of methods this view can handle.

class materializationengine.blueprints.materialize.api.SetBadRootsToNullResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str, table_name: str)[source]

Run workflow to lookup missing root ids and insert into database

Parameters:

datastack_name (str) – name of datastack from infoservice

class materializationengine.blueprints.materialize.api.TableResource(api=None, *args, **kwargs)[source]

Bases: Resource

get(aligned_volume_name, version)[source]
methods: Optional[List[str]] = {'GET'}

A list of methods this view can handle.

class materializationengine.blueprints.materialize.api.TestWorkflowResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(iterator_length: int = 50)[source]

Test workflow

Parameters:

iterator_length (int) – Number of parallel tasks to run. Default = 50

class materializationengine.blueprints.materialize.api.UpdateExpiredRootIdsResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str)[source]

Lookup root ids

Parameters:

datastack_name (str) – name of datastack from infoservice

class materializationengine.blueprints.materialize.api.UpdateLiveDatabaseResource(api=None, *args, **kwargs)[source]

Bases: Resource

methods: Optional[List[str]] = {'POST'}

A list of methods this view can handle.

post(datastack_name: str)[source]

Ingest new annotations and update expired root ids

Parameters:

datastack_name (str) – name of datastack from infoservice

class materializationengine.blueprints.materialize.api.VersionResource(api=None, *args, **kwargs)[source]

Bases: Resource

get(aligned_volume_name)[source]
methods: Optional[List[str]] = {'GET', 'POST'}

A list of methods this view can handle.

post(aligned_volume_name: str)[source]

Create an aligned volume database :param aligned_volume_name: name of aligned_volume from infoservice :type aligned_volume_name: str

materializationengine.blueprints.materialize.api.check_aligned_volume(aligned_volume)[source]