Making Autoprotocols more flexible

On my first attempt at translating an experimental protocol to the Autoprotocol format I got as far as creating a run on Transcriptic with the protocol which is awesome. The downside of that Autoprotocol though, was that the samples being used in the experiment were hardcoded into the python script, so any variation in experimental samples or parameters would have to be made in the python.

Thankfully the awesome people behind Autoprotocol made it possible to create a protocol that is parameterised. The Autoprotocol can be packaged up and uploaded to Transcriptic where Transcriptic’s web app generates a user interface making it easy for the user to just type in the experimental parameters and hit run as you can see in the screenshot.

How to package up an Autoprotocol python script

I created my assay package by leaning heavily on the protocols in Autoprotocol-core and the Transcriptic Runner documentation.

Change how the protocol is wrapped

The first thing to do was replace the start and the end of the Python script. At the start of old protocol there was a Protocol object where all the refs and actions get attached to.

import json
from autoprotocol.protocol import Protocol

p = Protocol()

# ...your protocol...

And at the end the whole the whole thing was dumped as a JSON object in Autoprotocol format:

# Builds the Autoprotocol JSON
print json.dumps(p.as_dict(), indent=2)

In the new protocol we don’t want the python script to dump JSON as it now has to operate slightly differently. Because of this we also don’t need to instantiate a Protocol object.

So the start now looks like this:

from autoprotocol.util import make_dottable_dict

def assay_name(protocol,params):
    params = make_dottable_dict(params)

    # ...your protocol...

In the new protocol every action is wrapped up inside a method assay_name in the example, that accepts the arguments protocol and params.

The end of the protocol now looks like this:

if __name__ == '__main__':
    from autoprotocol.harness import run
    run(assay_name, 'AssayName')

Let’s move on to the next task of parameterisation.

Remove hardcoded parameters

In the old protocol we defined a lot of fixed parameters, things like the type of container, what volume of sample, which sample, what wavelengths to make spectral measurements. All of this can be parameterised ultimately making the protocol more flexible and easier to use by colleagues who aren’t confident editing a python script.

In the old protocol the user didn’t have a choice of media I would always force the use of LB broth doped with ampicillin, because it was hardcoded into every dispense command. But what if a user wanted to use un-doped LB? Adding this flexibility is straight forward, by checking the experimental parameters passed to the protocol method and assigning the choice to a variable to use whenever media is needed. In the following example the choice between two types of media is made by checking the params object if simple conditional statements.

## Check the params object for the choice of media and make sure only one choice is made.
if params["media"]["lb-broth-100ug-ml-amp"] and not params["media"]["lb-broth-noAB"]:
    # Set the growth_media variable to the users choice.
    growth_media = "lb-broth-100ug-ml-amp"
elif params["media"]["lb-broth-noAB"] and not params["media"]["lb-broth-100ug-ml-amp"]:
    growth_media = "lb-broth-noAB"
else:
    ## Notify the user that they need to make a choice.
    raise RuntimeError("You must select a growth medium.")

protocol.dispense(96_well_container, growth_media, [{'column': 0}])

So where does the params object get populated?

Introducing Manifest.json

Manifest.json serves a couple of purposes. It is where the assignable parameters for the protocol are defined, in addition to a set of parameters that are used as defaults when we preview the protocol in testing with Transcriptic Runner.

This is the example manifest.json from the documentation:

{
  "version": "1.0.0",
  "format": "python",
  "license": "MIT",
  "protocols": [
    {
      "name": "SampleProtocol",
      "command_string": "python -m my_protocols.sample_protocol",
      "description": "this is a sample protocol",
      "preview": {
        "refs": {
          "sample_plate": {
            "type": "96-pcr",
            "discard": true
          }
        },
        "parameters": {
          "source_sample": "sample_plate/A1",
          "dest_sample": "sample_plate/A2",
          "transfer_vol": "5:microliter"
        }
      },
      "inputs": {
        "source_sample": "aliquot",
        "dest_sample": "aliquot",
        "transfer_vol": "volume"
      },
      "dependencies": []
    }
  ]
}

You can see it kicks off with some definitions around the version and other housekeeping details about the protocol. In the example there is just one protocol, however a whole array of protocols can be defined. Important bits here are:

1. command_string the path to the python script of the protocol
2. preview the parameters and refs used in the preview (good for testing locally)
3. inputs dictates the fields offered to the user to populate the params object

Let’s look at the inputs but for the growth media choice I mentioned earlier:

{
  "inputs": {
    "media": {
      "type": "group",
      "description": "Type of media to grow bacteria in. (check off only one)",
      "inputs": {
        "lb-broth-100ug-ml-amp": {
          "type": "bool"
        },
        "lb-broth-noAB": {
          "type": "bool"
        }
      }
    }
  }
}

Under the "media" property there are two inputs one for each medium, the "type" of input is bool indicating that the selection input is either true or false.

For this kind of input Transcriptic generates checkbox UI elements as seen below:

Inputs need to be added for each of the parameters you reference in the python protocol.

Directory Structure

Be sure to arrange the protocols in a file structure similar to that as recommended by Transcriptic:

protocols/
  manifest.json
  requirements.txt
  my_protocols/
    __init__.py
    sample_protocol.py

The manifest.json needs to be at the top level of the directory tree, so at least a level up from the python protocol.

Testing

To test the protocol using Transcriptic Runner in the same directory as the manifest.json run:

$ transcriptic preview AssayName

If all is working properly it dumps the JSON autoprotocol to STDOUT. Other wise you will get an error either due to the manifest.json or python protocol having errors. Keep fixing errors until you get the JSON dump, I found using pylint useful for fixing basic syntax errors in my python.

After that you can bounce it off of the Transcriptic servers by piping the JSON to analyze:

$ transcriptic preview AssayName | transcriptic analyze

I found it useful to create a run from the preview as I like to use the run UI on the Transcriptic web app to quickly scan through the run to make sure the protocol is doing what I want it to do.

$ transcriptic preview AssayName | transcriptic submit --project ":project_code" --title "Test Run" --test

If the run looks good on the Transcriptic web application it’s time to package it up.

Uploading releases to Transcriptic

Transcriptic has a really easy way of uploading packages of protocols to the server. In the directory create a .zip archive from the manifest.json and the directory containing the python protocol. Name the .zip release_someVersionNumber.zip in line with what version number is specified in the manifest.json.

Next login to the Transcriptic web app and click ‘Manage’ for your organization. Then click the ‘PACKAGES’ tab. From here click ‘Create New Package’. Name the protocol, give it a short description then upload the .zip file for the package. Click ‘Save & Analyze’. Under the ‘RELEASES’ you will want to click publish to ensure other members of your organization can use the protocol.

Whenever you add improvements to the protocol, bump the version number in the manifest.json, zip up all the files again and upload the new version, then hit publish! I think this release management is a really nice feature as version tracking of protocols is essential for experimental repeatability.

Summary

All in all the process of packaging up a protocol was not too difficult and I think it goes a long way to make the Transcriptic platform more widely accessible to all researchers, not just the ones that can write python. The ability to make protocols public is awesome as well. You could write a paper, publish it on PLOS, and reference your protocol on Transcriptic. Then when people want to try your technique, all they need to do is find your protocol on Transcriptic and use their samples! Think of what this will do for repeatability…

I had the pleasure of meeting Tali, Max and Dorothy-Lou from Transcriptic at SynBioBetaUK this week. They are all extremely smart and super friendly people, I highly recommend you get in touch with them if you are interested giving Transcriptic a try with your research.