Intent.Google.CloudStorage

The Google Cloud Storage module simplifies working with Google Cloud Storage in .NET applications by providing a robust C# client wrapper. This module includes an integration of the Google.Cloud.Storage.V1 NuGet package, facilitating direct interactions with Google Cloud Storage services.

Module Overview

Google Cloud Storage is a managed service designed for storing any amount of unstructured data and retrieving it as often as one likes. The Google Cloud Storage Module abstracts these interactions through the ICloudStorage interface, which defines methods for essential operations like upload, download, list, and delete. This interface ensures that the implementation details are encapsulated, allowing developers to focus on business logic rather than infrastructure management.

Practical Example

Consider a scenario within a corporate software system where it is necessary to store and fetch marketing materials from Google Cloud Storage. Below is an illustration of how the ICloudStorage interface can be implemented in a service class to manage these operations:

public class MarketingMaterialService
{
    private readonly ICloudStorage _cloudStorage;
    private readonly string _bucketName = "marketing-materials";

    public MarketingMaterialService(ICloudStorage cloudStorage)
    {
        _cloudStorage = cloudStorage;
    }

    public async Task<Uri> SaveMaterialAsync(string materialName, Stream content, string? contentType = null)
    {
        var materialUri = await _cloudStorage.UploadAsync(_bucketName, materialName, content, contentType);
        return materialUri;
    }

    public async Task<Stream> GetMaterialAsync(string materialName)
    {
        return  await _cloudStorage.DownloadAsync(_bucketName, materialName);
    }

    public async IAsyncEnumerable<Uri> ListMaterialsAsync()
    {
        List<Uri> materials = [];
        await foreach (var uri in _cloudStorage.ListAsync(_bucketName))
        {
            yield return uri;
        }
    }
}

This service class simplifies all functions related to storing and retrieving marketing materials, enhancing the manageability and cleanliness of the overall codebase.

Multitenancy Support

When used in conjunction with our Intent.Modules.AspNetCore.MultiTenancy module, has support for multitenancy. You can adjust the setting as follows:

Multiteanancy Setting

The Google Cloud Storage Data Isolation setting can be configured to either

None, Shared account (No multitenancy)
Separate Storage Account, Each tenant has their own configures Google Storage Account.

You simply need to configure your Cloud Storage Account details per tenant.

Here is a sample configuration for the In Memory tenant store.


public static void InitializeStore(IServiceProvider sp)
{
    var scopeServices = sp.CreateScope().ServiceProvider;
    var store = scopeServices.GetRequiredService<IMultiTenantStore<TenantExtendedInfo>>();

    store.TryAddAsync(new TenantExtendedInfo() { Id = "sample-tenant-1", Identifier = "tenant1", Name = "Tenant 1", GoogleCloudStorageConnection = "{Json Details Here}" }).Wait();
    store.TryAddAsync(new TenantExtendedInfo() { Id = "sample-tenant-2", Identifier = "tenant2", Name = "Tenant 2", GoogleCloudStorageConnection = "{Json Details Here}" }).Wait();
}

Pre-Signed Expiry Urls

Performing a GetAsync will give you back a link that you can use to gain access to an object for a limited amount of time (link).

To configure this expiry time you can add this entry in your appsettings.json file and specify the expiry duration as a TimeSpan.

"GCP": {
    "PreSignedUrlExpiry": "00:05:00",
    "CloudStorageAuthFileLocation": ""
  },

Authentication

THe module currently supports Google Service Account JSON key authentication. If additional authentication methods are required, the module can be updated to support these.

JSON key Creation

An example of how to create a Service Account JSON key is detailed below. The steps and process my differ based on your organisation's security requirements:

Log into your GCP console account

The first step is to create a Role which has permission to the Cloud Storage objects:

Search for Roles. This option is available under the IAM & Admin section
Once on the Roles screen, click + CREATE ROLE
Give the role a title (e.g. StorageFullAccess) and click + ADD PERMISSIONS
On the Add Permissions dialog which appears, under the Filter property name or value filter, enter storage.objects. This will list all permissions related to objects in storage.
The required permissions could be different based on your specific use case, but for the full functionality of this module to be leveraged, the following permisisons are required:
- storage.objects.create
- storage.objects.delete
- storage.objects.get
- storage.objects.list
- storage.objects.update
Click ADD on the dialog, and then click CREATE

A role has now been created. Next step is to create a Service Account to assign the role to.

Still under the IAM & Admin section:

Click Service Accounts (or search while in the GCP console)
Click + CREATE SERVICE ACCOUNT
Enter a Service Account Name (e.g. {ApplicationName}StorageAccount)
Click CREATE AND CONTINUE
From the Role dropdown, filter for and find the role create in the previous step
Click DONE
Once back to the Service Accounts List page, click on the newly created service account
Click KEYS and then ADD KEY => Create new key
Make sure JSON is selected and click CREATE
The JSON key file will be downloaded

Finally copy the JSON file from yours downloads folder to another folder of your choice, and make sure the CloudStorageAuthFileLocation setting in your applications appsettings.json is updated to the location of the key file:

"GCP": {
    "PreSignedUrlExpiry": "00:05:00",
    "CloudStorageAuthFileLocation": "C:\\gcpkeys\\my-project-183005-10ed20d64009.json"
  },

Now when using ICloudStorage the key file will be used to authenticate with Google Cloud Storage.

Ensure to create the necessary bucket(s) in Google Cloud Storage prior to performing any operations with them.