Introduction

Welcome to the Gencove developer docs!

Gencove is enabling users to do more with their genomic data by uploading their existing genomic data files from AncestryDNA, 23andMe, etc. and using apps built on top of the Gencove platform.

In case users do not already have their data, they can easily get it by purchasing a Gencove kit.

Developers can create apps that use genomic data in Gencove using our API. Access to invididual user data is requested by providing the user with a unique app URL: app_url.

Once access has been granted by the user, the developer only needs their API key (available in user profile) and a few simple API calls.

The data developers have access to is:

.vcf file with imputed genotypes (using the 1000 Genomes Phase 3 reference panel)
The original raw data file, which is either:
- AncestryDNA, 23andMe, etc. data file
- .bam file with raw sequencing reads (in case the user ordered a Gencove kit)

Try out the examples on the side and read on to see how you can use our API to start an app and get users on board in no-time.

Also, the full API reference for publicly available endpoints is available here: API reference

Using the API

Create an app

To create an app:

Create a free Gencove account on the developer website
Create an app by clicking “Create an new app” or going to My Apps -> Create a new app

Authentication

# API key
import requests

response = requests.get(
    'https://rest.gencove.com/welcome-api-key',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    }
)

# API key
$ curl -H 'Authorization: GENCOVE-API-KEY <your_api_key>'\
    https://rest.gencove.com/welcome-api-key

The Gencove API has 2 main types of keys:

API key, used for programmatic access to Gencove API endpoints
- The API is accessed by placing your API key in the Authorization header of HTTP requests: Authorization: GENCOVE-API-KEY <api_key>
App signing keys, used for signing the JWTs in app_urls given to users
- Every app has a separate key

Accessing user data

Method 1: Direct URLs

App URL diagram

This method assumes the user starts their interaction with your app on the Gencove website. The developer defines a direct_url, which may contain the following tokens:

{vcf} - a presigned URL pointing to the user’s VCF file
{tbi} - a presigned URL pointing to the user’s .tbi index for the VCF file
{csi} - a presigned URL pointing to the user’s .csi index for the VCF file
{external_id} - the unique external identifier generated for the user’s membership in the app, which is used by the app to access the user’s data in their Gencove account.

Upon discovering the app on the Gencove website, the user is taken to a confirmation page and routed to the specified direct_url. If the URL contains any of the above tokens, they are replaced with the corresponding data.

The developer may choose whether the URLs that replace the tokens are URL-encoded or not. We highly recommend using URL encoding.

The developer may also define a member_url, which is used to route the user from the Gencove website after they have provided access to their data. This is in contrast to the direct_url, where the user is sent before providing access to their data. It can be used in case the developer would like to provide a different experience for users that have already provided their data. To enable a personalized experience, the member_url may also contain an {external_id} token, which will be replaced by the user’s external_id.

If both the direct_url and landing_url (see App URLs below) are defined, the direct_url takes precedence.

Method 2: App URLs

Direct URL diagram

Create app_url offline with app signing key

# Assuming PyJWT package is installed
import jwt

token = jwt.encode(
    {
        'app_id': <app_id>,
        'external_id': '<external_id>',
        'subsidy_in_cents': <subsidy>
    }, 
    '<app_signing_key>', 
    algorithm='HS256'
)

app_url = 'https://dl.gencove.com/connect/{}'.format(token)

# Assuming PyJWT Python package is installed
$ TOKEN=$(pyjwt --key=<app_signing_key> encode\
    app_id=<app_id>\
    external_id=<external_id>\
    subsidy_in_cents=<subsidy>)

$ APP_URL="https://dl.gencove.com/connect/$TOKEN"

Create app_url using Gencove API

# Assuming requests package is installed
import requests

response = requests.post(
    'https://rest.gencove.com/apps/jwt',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    },
    json={
        'jwt_data': [
            {
                'app_id': <app_id>,
                'external_id': '<external_id>',
                'subsidy_in_cents': <subsidy>
            }
        ]
    }
)

app_url = response['jwt_data'][0]['href']

# Assuming cURL is installed
$ curl -H "Authorization: GENCOVE-API-KEY <your_api_key>"\
    -H "Content-Type: application/json"\
    -X POST\
    -d '{"jwt_data": [{"app_id": <app_id>, "external_id": "<external_id>", "subsidy_in_cents": <subsidy>}]}'\
    https://rest.gencove.com/apps/jwt

This method assumes the user starts their interaction with your app on the app landing page (landing_url), where they complete basic onboarding, consent, and any other step prior to providing access to data in their Gencove account.

In case the user discovers your app on the Gencove website (e.g., the app dashboard), they are routed to the app landing page provided in the app settings.

The developer requests access to user data by generating a unique app_url for each user and providing it to the user. Once the user visits the app_url and approves, the developer immediately gains access to the data.

Developers can generate app_urls in one of 3 ways:

Using the developer website, by going to My Apps -> (App name) -> Generate links
Offline, using standard JSON Web Token (JWT) libraries
Online, using a helper endpoint on the Gencove API

The structure of the app_url is: https://dl.gencove.com/connect/<JWT>

The main part of the app_url is the JWT, which contains data about the app (app_id) and a unique identifier for the user (external_id). Gencove does not enforce the uniqueness of external_id within an app.

The JWT may also contain an optional subsidy (subsidy_in_cents), which allows the developer to discount Gencove kits for users.

The JWT is signed with the app signing key, which can be obtained from the developer website under My apps -> (app name) -> Show/hide link signing key.

The app_url will take the user to the Gencove website to approve access to their data. In case the user has the Gencove app installed on their device, the app_url will act as a deeplink and open the Gencove app to complete the approval process.

app_url with continue_url

app_url_with_continue = '{}?continue_url={}'.format(app_url, continue_url)

APP_URL_WITH_CONTINUE="$APP_URL?continue_url=$CONTINUE_URL"

Developers may optionally:

append a continue_url parameter to the app_url, which is used as the last step of the data access approval flow in order to provide a path forward outside of Gencove once the user approves access (see figure above). For example, this can be used for completing an onboarding process in your app. continue_url must be URL-encoded.
define a member_url, which is used as a customized link for sending the user to your app throughout the Gencove website after your app has access to user data (see figure above). This is in contrast to landing_url (used before providing access to their data) and continue_url (only used in the data access approval flow). It can be used in case the developer would like to provide a different experience for users arriving from the Gencove website that have already provided their data. To enable a personalized experience, the member_url may also contain an {external_id} token, which will be replaced by the user’s external_id.

It is useful attribute a unique identifier for each user in order to be able to facilitate dereferencing the user data once it becomes available. In case you do not already have a unique id scheme for users, we recommend using UUIDv4.

Webhooks

Webhook diagram

{
    "event": "app_member_join",
    "data": {
          "app_id": 1,
          "member_id": 2,
          "external_id": "member_100003",
          "subsidy_in_cents": 0,
          "participating": true,
          "jwt_log": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhcHBfaWQiOjEsImV4dGVybmFsX2lkIjoibWVtYmVyXzEwMDAwMSIsInN1YnNpZHlfaW5fY2VudHMiOjB9.TR4FVXDxdD0EL-LaHOaoOKQhv1N8UN0eYdvlt3nQ5ao"
    }
}

The developer can also define a webhook URL, where app events are submitted via HTTP POST requests. The content of the webhook contains:

Type of event, event
Data describing the event in more detail, data

Currently, there are 4 types of events:

app_member_join: generated when a user approves the app for access to their data by using the app_url
app_member_leave: generated when a user revokes access to their data via profile settings
app_member_rejoin: generated when a user re-approves access to their data via profile settings
app_member_results_ready: generated when user data becomes available when it was not available before. This webhook is only generated when the app is approved for access to user data.
test: generated when validating the webhook URL. If a webhook_url is defined when creating or updating an app, the Gencove backend will attempt to validate the URL by generating a HTTP POST request with this event type.

Webhooks are only active when the app is in the enabled state.

In general, user data availability (and updates) should be checked on app_member_join, app_member_rejoin and app_member_results_ready webhooks.

Listing your apps

import requests

response = requests.get(
    'https://rest.gencove.com/apps',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    }
)

$ curl -H "Authorization: GENCOVE-API-KEY <your_api_key>"\
    https://rest.gencove.com/apps

Your apps can be listed with a GET HTTP request to /apps

Info about a single app

import requests

response = requests.get(
    'https://rest.gencove.com/apps/<app_id>',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    }
)

response = requests.get(
    'https://rest.gencove.com/apps/<app_id>/stats',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    }
)

$ curl -H "Authorization: GENCOVE-API-KEY <your_api_key>"\
    https://rest.gencove.com/apps/<app_id>

$ curl -H "Authorization: GENCOVE-API-KEY <your_api_key>"\
    https://rest.gencove.com/apps/<app_id>/stats

Data and stats for a specific app can be listed with GET HTTP requests to /apps/<app_id> and /apps/<app_id>/stats, respectively.

Listing app users

import requests

response = requests.get(
    'https://rest.gencove.com/apps/<app_id>/members',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    }
)

$ curl -H "Authorization: GENCOVE-API-KEY <your_api_key>"\
    https://rest.gencove.com/apps/<app_id>/members

App users can be listed with a GET HTTP request to /apps/<app_id>/members

Info about a specific user

import requests

response = requests.get(
    'https://rest.gencove.com/apps/<app_id>/members/<member_id>',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    }
)

$ curl -H "Authorization: GENCOVE-API-KEY <your_api_key>"\
    https://rest.gencove.com/apps/<app_id>/members/<member_id>

Info about a single user can be listed with a GET HTTP request to /apps/<app_id>/members/<member_id>

Accessing user genomic data

import requests

response = requests.post(
    'https://rest.gencove.com/apps/<int:app_id>/raw-data',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    },
    json={
        'member_ids': [<member_id_1>, <member_id_2>, ..., <member_id_N>]
    }
)

response = requests.post(
    'https://rest.gencove.com/apps/<int:app_id>/raw-data',
    headers={
        'Authorization': 'GENCOVE-API-KEY <your_api_key>'
    },
    json={
        'external_ids': [<external_id_1>, <external_id_2>, ..., <external_id_N>]
    }
)

$ curl -H "Authorization: GENCOVE-API-KEY <your_api_key>"\
    -H "Content-Type: application/json"\
    -X POST\
    -d '{"member_ids": [<member_id_1>, <member_id_2>, ..., <member_id_N>]}'\
    https://rest.gencove.com/apps/<int:app_id>/raw-data

$ curl -H "Authorization: GENCOVE-API-KEY <your_api_key>"\
    -H "Content-Type: application/json"\
    -X POST\
    -d '{"external_ids": [<external_id_1>, <external_id_2>, ..., <external_id_N>]}'\
    https://rest.gencove.com/apps/<int:app_id>/raw-data

Users’ genomic data can be accessed by POSTing a list of member_ids or external_ids to /apps/<int:app_id>/raw-data .

The endpoint will return a list of objects containing the following data for each member_id or external_id:

member_id - Gencove member id (unique)
external_id - the external id provided in JWT when requesting access to user data
app_id - the app id
vcf_url_s3 - a presigned URL for downloading the imputed .vcf file
snp_url_s3 - a presigned URL for downloading the original AncestryDNA, 23andMe, etc. data file
bam_url_s3 - a presigned URL for downloading the raw .bam file (if the user got their genomic data through Gencove)
bai_url_s3 - a presigned URL for downloading the .bam file index
fastq_nongrch37_url_s3 - a presigned URL for downloading the .fastq file with non-human reads
ancestry_url_s3 - a presigned URL for downloading a .json file with the output of Gencove’s ancestry analysis
microbiome_url_s3 - a presigned URL for downloading a .json file with the output of Gencove’s microbiome analysis

The presigned download URLs expire after 2 days, but there is not a limit on the number of generated presigned URLs.

In case the respective id is not found or does not yet have data delivered, it will be skipped

Gencove CLI

The Gencove CLI can be used to access user genomic data in bulk.

Installation

# In shell:
pip install gencove

$ pip install gencove

The Gencove CLI can be easily installed using the Python package manager pip. The source code is available on GitLab.

Usage

# In shell (standard invocation)
gencove app raw_data <app-id> <external-ids-file> <csv-file>

# Using stdin and stdout:
cat <member-ids-file> | gencove app raw_data --output-format json --id-type member <app-id> - - > <json-file>

# Standard invocation:
$ gencove app raw_data <app-id> <external-ids-file> <csv-file>

# Using stdin and stdout:
$ cat <member-ids-file> | gencove app raw_data --output-format json --id-type member <app-id> - - > <json-file>

The Gencove CLI enables exporting user data in bulk to csv or json format.

The API key can be provided in the environment variable $GENCOVE_API_KEY or inline via the --api-key option.

Input and output can be a file path or “-” for stdin and stdout.

The input stream is expected to be one id per line.

If using the default “external” id type, multiple entries for the same external id may be returned, since it is not guaranteed to be unique. Member (i.e., Gencove) ids are guaranteed to be unique.

In case data is not available for a given id or the id is not found, it is skipped.

Order of ids from input is not guaranteed to be preserved in output.

# In shell (get help)
gencove app raw_data --help

# Get help
$ gencove app raw_data --help

For detailed usage instructions, use the --help flag

Testing environment

Developers may use the Gencove staging environment for development and testing.

The staging developer website URL is: https://app-staging.gencove.com

The staging API URL is: https://rest-staging.gencove.com

API Reference

The full API reference for publicly available endpoints is available here: API reference

Terms

We reserve the right to remove your access to our API for any reason at our sole discretion. We will provide a more detailed description of our expectations in due course. However, overall we expect that:

you will not mislead users into disclosing their data under false pretenses
you will have a clear policy on how users can opt out of your use of their data should they change their minds about sharing it with you.

FAQ

User FAQ