Acceptance Testing
Manifold makes it easy for providers to validate their integration with a command line tool we call Grafton. Grafton is used for local development of a Provider's implementation of our Provider API endpoints.
Grafton works by standing in for the Manifold API and sends various requests to your in-progress integration. These tests include: resource and credential provisioning, resizing, deprovisioning, and single sign-on requests. To catch implementation errors, Grafton not only tests the success path but also an integration's validation and error handling.
While the tests are running through Grafton, a local version of Manifold's Connector API is also run for retrieving user or resource data as well as completing asynchronous operations.
Grafton is open source with releases available on GitHub.
We appreciate and welcome all feedback as you use our development tooling.
The latest installation instructions are documented in Grafton's GitHub repository.
The first step to use Grafton is to generate a test master key which is used for signing requests handled by your integration.
When testing your integration, Grafton authenticates using this test master key. When running in production, Manifold will authenticate using our production public key.
$ grafton generateGenerating Master KeypairWriting master keypair to file: masterkey.jsonSuccess.​$ cat masterkey.json{# Use the public_key in your integration"public_key": "bBbFBCDG8UEqSgiygslX2TZP/RUCD0GTHh43nXFxx1U=",# Grafton will use the private_key to sign requests"private_key": "s/4ZT/a4BXBrffQdC7bbW+NalAVc7THEuctqeruRVj1sFsUEIMbxQSpKCLKCyVfZNk/9FQIPQZMeHjedcXHHVQ=="}
Use the public master key contained in the generated file to validate requests created by Grafton during testing. Once integrated and in production, you must be using Manifold's Master Public Signing Key to validate requests coming from the Manifold servers.
To make requests against the locally run Connector API served by Grafton, you can use the following set of test credentials:
CLIENT_ID=21jtaatqj8y5t0kctb2ejr6jev5w8CLIENT_SECRET=3yTKSiJ6f5V5Bq-kWF0hmdrEUep3m3HKPTcPX7CdBZw
These values will be consumed by your application, and must also be supplied to the Grafton test command, as shown:
$ grafton test \--client-id=21jtaatqj8y5t0kctb2ejr6jev5w8 \--client-secret=3yTKSiJ6f5V5Bq-kWF0hmdrEUep3m3HKPTcPX7CdBZw \... # other command line flags
Once you've configured your integration with the generated signing key and, optionally, with OAuth credentials, you can begin testing with Grafton.
You can invoke the test command with parameters representing how a user may be configuring your product on Manifold. Typically you need to provide the product, plan, and a region.
In this example we look at resizing a resource. We start with a resource of the plan small from the bear product and are resizing to the large plan:
$ grafton test \--product=bear \ # Product label--plan=small \ # Plan label--new-plan=large \ # Plan label to resize to once the resource has been provsisioned--plan-features='{"age":2,"hat_color":"red"}' \ # Features of the plan--region=aws::us-east-1 \ # Region label--client-id=21jtaatqj8y5t0kctb2ejr6jev5w8 \ # OAuth Credential (Optional)--client-secret=3yTKSiJ6f5V5Bq-kWF0hmdrEUep3m3HKPTcPX7CdBZw \ # OAuth Credential (Optional)--connector-port=3000 \ # Port used by Grafton to serve Connector API (Optional)http://localhost:4567/v1 # URL to your integration
Green checkmarks will be shown once for each passing test as shown here:
Use the --help flag to see all available Grafton options:
$ grafton --help​NAME:grafton - Tool for testing integrations with Manifold​USAGE:grafton [global options] command [command options] [arguments...]​VERSION:0.15.0​COMMANDS:credentials Manage OAuth 2 credential pairs for Manifold.cogenerate Generates public and private signing keys for testing Manifold API integrations locallyserve Serves a local version of of the Connector APItest Tests the API endpoints required to integrate with Manifoldversion Display version of utilityhelp, h Shows a list of commands or help for one command​GLOBAL OPTIONS:--help, -h show help--version, -v print the version
Here is an example of viewing the parameters of the test command:
$ grafton test --help​NAME:grafton test - Tests the API endpoints required to integrate with Manifold​USAGE:grafton test [command options] [url]​OPTIONS:--product value The label of the product being provisioned [$PRODUCT]--plan value The label of the plan for the provisioning resource [$PLAN]--plan-features value A JSON object describing the selected features for the provisioning resource [$PLAN_FEATURES]--new-plan value The plan to resize the instance to from the original plan [$NEW_PLAN]--new-plan-features value A JSON object describing the selected features for the resizing from the original plan [$NEW_PLAN_FEATURES]--region value The label of the region which the resource will be provision in [$REGION]--import-code value The import code to import an existing resource for that resource [$IMPORT_CODE]--exclude value Exclude running these feature tests (and those that depend on it) [$EXCLUDE]--no-error-cases Skip running the error case tests [$NO_ERROR_CASES]--log value Informational logging level during tests. One of (off, info, verbose) (default: "off") [$LOG]--client-id value Client ID to use for SSO and local Connector API testing [$OAUTH2_CLIENT_ID]--client-secret value Client secret to use for SSO and local Connector API testing [$OAUTH2_CLIENT_SECRET]--connector-port value Local port for running the fake Connector API for SSO and Async testing (default: 0) [$CONNECTOR_PORT]--callback-timeout value duration to wait (max. 24hours) for a callback (default: 5m) [$CALLBACK_TIMEOUT]--resource-measures value Optional measures map to be returned by resource measures (default: "{\"feature-a\": 0, \"feature-b\": 1000}")--credential value Describes the credential type that is supported by this product. One of (single, multiple) (default: "multiple")
