Cloud Data Fusion Private Instance Guide

Pricing

  • Cloud Data Fusion instance hours to operate the data integration interface
  • Cloud Dataproc VMs to execute the transformations prescribed by Cloud Data Fusion

How to Create a Private Instance

  1. Go to the VPC Network page of your network in which you want to create private Cloud Data Fusion Instance.
  2. Click on the Private Service Connection tab.
  3. If asked, enable Service Networking API.
  4. Allocate an IP range of size /22 by clicking on the Allocate IP Range button.

Command to create an instance

curl -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json" https://$DATA_FUSION_API_NAME/v1beta1/projects/$PROJECT/locations/ $LOCATION/instances?instanceId=<INSTANCE_NAME> -X POST -d '{"description": "Private CDF instance created through REST.", "type": "BASIC", "privateInstance": true, "networkConfig": {"network": "VPC_NETWORK", "ipAllocation": "IP_RANGE"}}'

Peering With Cloud Data Fusion Network

Creating VPC Peering

  1. Go to your VPC Network
  2. Select VPC Network Peering
  3. Click on Create Connection
  4. Give a name to your peering ex: datafusion-peering
  5. Make sure that Your VPC network lists the network which you selected while creating Cloud Data Fusion instance.
  6. In Peered VPC network select In another project Provide the tenant project ID in the Project ID field
  7. In VPC network name provide <instance-region>-<instanceid>. Please note the network
  8. name in tenant project is of the format <instance-region>-<instanceid> i.e why you are providing the above name.
    Click on Exchange custom routes and select both Import custom routes (so you can access CDF UI) and Export custom routes (so CDF can access on-prem connected to your VPC network)
  9. Click on Create and wait for the operation to complete.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Harshad Patel

Harshad Patel

15 Followers

7x GCP | 2X Oracle Cloud| 1X Azure Certified | Cloud Data Engineer