Importing and Exporting data from Dgraph Cloud

We’re overhauling Dgraph’s docs to make them clearer and more approachable. If you notice any issues during this transition or have suggestions, please let us know.

Importing data into Dgraph Cloud

1. Obtain dgraph binary or the latest docker image by following the installation instructions. This is required to run Dgraph CLI command dgraph live.

2. Obtain the gRPC endpoint of your Dgraph Cloud backend and a valid Client API key.

   An administrator gets those information with the following steps:

   • Log into the Dgraph Cloud account, select the backend

   • In the Admin section of the Dgraph Cloud console, go to Settings and copy the value of the gRPC Endpoint from the General tab.

   • Access the API Keys tab to generate an Client API Key.

     The gRPC endpoint is different from the GraphQL endpoint. The gRPC endpoint looks like <domain>:443

3. Run the Live Loader as follows:

• Docker
• Local

docker run -it --rm -v <local-path-to-data>:/tmp dgraph/dgraph:latest \
dgraph live --slash_grpc_endpoint <grpc-endpoint> -f /tmp/<data-file> -s /tmp/<schema-file> -t <api-key>

Load multiple data files by using

docker run -it --rm -v <local-path-to-data>:/tmp dgraph/dgraph:latest \
  dgraph live --slash_grpc_endpoint <grpc-endpoint> -f /tmp -s /tmp/<schema-file> -t <api-key>

When the path provided with -f, --files option is a directory, then all files ending in .rdf, .rdf.gz, .json, and .json.gz will be loaded. Be sure that your schema file has another extension (.txt or .schema for example).

Exporting data from Dgraph Cloud

You can export your data as an Administrator from one Dgraph Cloud backend, and then import this data back into another Dgraph instance or Dgraph Cloud backend. You can also export data from Dgraph Cloud using the Dgraph Cloud API. For more information, see Cloud API documentation.

To import data to Dgraph Cloud, see Live Loader.

As an Administrator you can export data from a Dgraph Cloud shared instance or dedicated instance. On a dedicated instance with multi-tenancy feature enabled you can export data across the cluster, or a specific namespace depending on the type of administrative privileges you have.

Exporting data from Dgraph Cloud using the console

1. In the Admin section of the Dgraph Cloud console, go to Settings.
2. In the Exports tab, click Create Export.
3. In the New export dialog, select the format you want to export.
4. Click Create.

Depending on the format that you chose to create an export, three files are generated.

Ensure that you download these files as soon as possible because the links to download these files expire after 48 hours from the time they were generated.

Exporting data from Dgraph Cloud using a GraphQL client

1. Generate API Key for authentication.

2. Make a note of the GraphQL endpoint for the instance from Overview in the Dgraph Cloud console. Replace /graphql with /admin/slashin the GraphQL endpoint to get the <ADMIN_ENDPOINT>.

3. Authenticate the admin API requests by adding the <APIKEY> as the Dg-Auth header to every HTTP request.

4. To export data you need to send authenticated request to <ADMIN_ENDPOINT>.

5. Export data in JSON or RDF <FORMAT> using this mutation:

   mutation {
     export(format: "<FORMAT>") {
       response {
         message
         code
       }
       exportId
       taskId
     }
   }
   

   A response similar to this appears:

       "data": {
          "export":"exports/2011-12-08/0x18986fd-558223708",
          "response": {
             "code": "Success",
             "message": "Export queued with ID 0x9d2e13e8a"
          },
          "taskID": "0x9d2e13e8a"
       }
      }
   }
   

6. Make a note of the <EXPORT_ID> and the <TASK_ID>.

7. To get the status of export and the signed URLs to download the exported files, use this mutation:

   query {
     exportStatus(exportId: "<EXPORT_ID>", taskId: "<TASK_ID>") {
       kind
       lastUpdated
       signedUrls
       status
     }
   }
   

   Depending on the format that you chose to create an export, three files are generated.

Ensure that you download these files as soon as possible because the signed URLs to download these files expire after 48 hours from the time they were generated. You can use curl -O <SIGNED_URL> to download the files to the current directory.

Exporting data from Dgraph Cloud programmatically

You can also export data from Dgraph Cloud programmatically using the Dgraph Cloud API. For more information, see Cloud API documentation.

Exporting data with multi-tenancy feature enabled in Dgraph Cloud

With Multi-Tenancy feature enabled, for any GraphQL request you need to provide the accessJWT for the specific user in the X-Dgraph-AccessToken header.

You can trigger two types of exports:

• Cluster-wide export: this is an export of the entire backend (including all namespaces). This request can be only triggered by the Guardian of Galaxy users.
• Namespace-specific export: this is an export of a specific namespace. This request can be triggered by the Guardian of Galaxy users and by the Guardian of Namespace users.

Cluster-wide exports

This can only be done by the Guardian of Galaxy users (super admin), the steps are:

1. Get the accessJWT token for the Guardian of Galaxy user. Send the following GraphQL mutation to the /admin endpoint:

   mutation login($userId: String, $password: String, $namespace: Int) {
     login(userId: $userId, password: $password, namespace: $namespace) {
       response {
         accessJWT
         refreshJWT
       }
     }
   }
   

   Your variables should be referring to the Guardian of Galaxy user:

   {
     "userId": "groot",
     "password": "password",
     "namespace": 0
   }
   

2. Once obtained the accessJWT token you need to pass it in X-Dgraph-AccessToken Header and only then you can send the following GraphQL mutation to /admin/slash endpoint:

   mutation {
     export(namespace: -1) {
       response {
         code
         message
       }
       exportId
       taskId
     }
   }
   

3. Once done, you can now send the following GraqhQL mutation to get the signedUrls from where you can download your export files:

   query {
     exportStatus(
       exportId: "<paste-your-exportId>"
       taskId: "<paste-your-taskId>"
     ) {
       kind
       lastUpdated
       signedUrls
       status
     }
   }
   

Namespace-specific exports

Namespace-specific exports can be triggered by the Guardian of Galaxy users. In this case you can follow the same steps for the Cluster-wide exports and replace the namespace value from -1 to the namespace you want to export. It is important that you get the accessJWT token for the Guardian of Galaxy user and pass it in the X-Dgraph-AccessToken header.

For example, if you want to export the namespace 0x123 your GraphQL request sent to the /admin/slash endpoint would look like:

mutation {
  export(namespace: 123) {
    response {
      code
      message
    }
    exportId
    taskId
  }
}

You can also trigger namespace-specific export using the Guardian of Namespace users, in this case there is no need to specify any namespace in the GraphQL request as these users can only export their own namespace. It is important that you get the accessJWT token for the Guardian of Namespace user and pass it in the X-Dgraph-AccessToken header.

The GraphQL request sent to the /admin/slash endpoint would be:

mutation {
  export {
    response {
      code
      message
    }
    exportId
    taskId
  }
}