Home   >   GCP   >   Labs - Learning Path - GCP Professional Data Engineer Certification   >   Using Storage Transfer Service in GCS
some

Using Storage Transfer Service in GCS

Lab Details:

  1. This lab walks you through using Google Cloud Storage Transfer Service

  2. Region: us-central1

  3. Duration: 45 minutes

Note: Do not refresh the page after you click Start Lab, wait for a few seconds to get the credentials.
In case while login into Google, if it asks for verification, please enter your mobile number and verify with OTP, Don't worry this Google Account will be deleted after the lab.

What is Cloud Storage Bucket?

Google Cloud Storage is an object-based service that allows you to store objects in any file format. An Object can be any individual file or folder whereas a bucket is referred to as a container or object-based drive or a root folder where you store your objects.

Cloud Storage can be used as a staging area also. For example, if any of your applications need to access the data from somewhere like Cloud SQL, BigQuery, or Dataproc, you can place the data in a cloud storage bucket.

There are many ways for accessing your objects and Buckets.

The first and easiest way of accessing Cloud Storage is using GCP Console.

Another way can be using Command-Line or SDK. Before opting, you have to install the SDK in the system.

You can also access it using RESTful APIs to HTTP URLs.

You can choose to create different location types in which Multi-regional is where you would store files for use by your applications which are worldwide. Dual-region is best when your file needs to be accessed in associated regions. Regional is best for any internal jobs that require storage.

What is a Google Cloud Storage Transfer Service?

Storage Transfer Service is a fully managed, code-free, highly scalable service to automate transfers from other public clouds into Cloud Storage or from On-Premise Datacenters to Google Cloud. You just have to define Source and Destination, the rest of Google Cloud will take care of it.

  • For Storage Transfer Service for Cloud-to-Cloud transfers supports transfers into Cloud Storage from Amazon S3, Azure Blob Storage, within Google Cloud Buckets, and HTTP/HTTPS Locations.
  • For Storage Transfer Service for data transfers for on-premises data transfers, you need to install an agent on Datacenter VMs/Machines and create a Job for transfers.

Storage Transfer Service vs Transfer Appliance  :

The main difference between both the services depends on the below factors:

  1. Amount of Data to be Transferred.
  2. Current Location of Data (on-Prem/Cloud).
  3. The bandwidth of Internet Connectivity.
  4. Maximum Time required for Migration.

In general, if the data is huge i.e. greater than 10TB and you have less internet connectivity go for Storage appliance else go for Storage Transfer Service or gsutil . 

You can refer to the below chart while thinking to transfer data to Cloud :

Lab Tasks:

  1. Creating Source and Destination Bucket.

  2. Uploading an Object in the Source bucket.

  3. Creating a Storage Transfer Service Job.

  4. Verifying the successful execution of the Job.

 



Join Whizlabs_Hands-On to Read the Rest of this Lab..and More!

Step 1 : Login to My-Account
Step 2 : Click on "Access Now" to view the course you have purchased
Step 3 : You will be taken to our Learn Management Solution (LMS) to access your Labs,Quiz and Video courses

Open Console