Acceptance Testing

Manifold makes it easy for providers to validate their integration with a command line tool we call Grafton. Grafton is used for local development of a Provider's implementation of our Provider API endpoints.

Acceptance testing data flow

Grafton works by standing in for the Manifold API and sends various requests to your in-progress integration. These tests include: resource and credential provisioning, resizing, deprovisioning, and single sign-on requests. To catch implementation errors, Grafton not only tests the success path but also an integration's validation and error handling.

While the tests are running through Grafton, a local version of Manifold's Connector API is also run for retrieving user or resource data as well as completing asynchronous operations.

Grafton is open source with releases available on GitHub.

We appreciate and welcome all feedback as you use our development tooling.


The latest installation instructions are documented in Grafton's GitHub repository.


Authenticating Manifold requests

The first step to use Grafton is to generate a test master key which is used for signing requests handled by your integration.

When testing your integration, Grafton authenticates using this test master key. When running in production, Manifold will authenticate using our production public key.

$ grafton generate
Generating Master Keypair
Writing master keypair to file: masterkey.json
$ cat masterkey.json
# Use the public_key in your integration
"public_key": "bBbFBCDG8UEqSgiygslX2TZP/RUCD0GTHh43nXFxx1U=",
# Grafton will use the private_key to sign requests
"private_key": "s/4ZT/a4BXBrffQdC7bbW+NalAVc7THEuctqeruRVj1sFsUEIMbxQSpKCLKCyVfZNk/9FQIPQZMeHjedcXHHVQ=="

Use the public master key contained in the generated file to validate requests created by Grafton during testing. Once integrated and in production, you must be using Manifold's Master Public Signing Key to validate requests coming from the Manifold servers.

Connector API (Optional)

To make requests against the locally run Connector API served by Grafton, you can use the following set of test credentials:


These values will be consumed by your application, and must also be supplied to the Grafton test command, as shown:

$ grafton test \
--client-id=21jtaatqj8y5t0kctb2ejr6jev5w8 \
--client-secret=3yTKSiJ6f5V5Bq-kWF0hmdrEUep3m3HKPTcPX7CdBZw \
... # other command line flags

Running Tests

Once you've configured your integration with the generated signing key and, optionally, with OAuth credentials, you can begin testing with Grafton.

You can invoke the test command with parameters representing how a user may be configuring your product on Manifold. Typically you need to provide the product, plan, and a region.

In this example we look at resizing a resource. We start with a resource of the plan small from the bear product and are resizing to the large plan:

$ grafton test \
--product=bear \ # Product label
--plan=small \ # Plan label
--new-plan=large \ # Plan label to resize to once the resource has been provsisioned
--plan-features='{"age":2,"hat_color":"red"}' \ # Features of the plan
--region=aws::us-east-1 \ # Region label
--client-id=21jtaatqj8y5t0kctb2ejr6jev5w8 \ # OAuth Credential (Optional)
--client-secret=3yTKSiJ6f5V5Bq-kWF0hmdrEUep3m3HKPTcPX7CdBZw \ # OAuth Credential (Optional)
--connector-port=3000 \ # Port used by Grafton to serve Connector API (Optional)
http://localhost:4567/v1 # URL to your integration

Green checkmarks will be shown once for each passing test as shown here:

Grafton output example


Use the --help flag to see all available Grafton options:

$ grafton --help
grafton - Tool for testing integrations with Manifold
grafton [global options] command [command options] [arguments...]
credentials Manage OAuth 2 credential pairs for
generate Generates public and private signing keys for testing Manifold API integrations locally
serve Serves a local version of of the Connector API
test Tests the API endpoints required to integrate with Manifold
version Display version of utility
help, h Shows a list of commands or help for one command
--help, -h show help
--version, -v print the version

Here is an example of viewing the parameters of the test command:

$ grafton test --help
grafton test - Tests the API endpoints required to integrate with Manifold
grafton test [command options] [url]
--product value The label of the product being provisioned [$PRODUCT]
--plan value The label of the plan for the provisioning resource [$PLAN]
--plan-features value A JSON object describing the selected features for the provisioning resource [$PLAN_FEATURES]
--new-plan value The plan to resize the instance to from the original plan [$NEW_PLAN]
--new-plan-features value A JSON object describing the selected features for the resizing from the original plan [$NEW_PLAN_FEATURES]
--region value The label of the region which the resource will be provision in [$REGION]
--import-code value The import code to import an existing resource for that resource [$IMPORT_CODE]
--exclude value Exclude running these feature tests (and those that depend on it) [$EXCLUDE]
--no-error-cases Skip running the error case tests [$NO_ERROR_CASES]
--log value Informational logging level during tests. One of (off, info, verbose) (default: "off") [$LOG]
--client-id value Client ID to use for SSO and local Connector API testing [$OAUTH2_CLIENT_ID]
--client-secret value Client secret to use for SSO and local Connector API testing [$OAUTH2_CLIENT_SECRET]
--connector-port value Local port for running the fake Connector API for SSO and Async testing (default: 0) [$CONNECTOR_PORT]
--callback-timeout value duration to wait (max. 24hours) for a callback (default: 5m) [$CALLBACK_TIMEOUT]
--resource-measures value Optional measures map to be returned by resource measures (default: "{\"feature-a\": 0, \"feature-b\": 1000}")
--credential value Describes the credential type that is supported by this product. One of (single, multiple) (default: "multiple")