Google Cloud BigQuery Storage

The BigQuery Storage Read API provides fast, high-throughput access to BigQuery table data. The gRPC API is exposed through the google.cloud.bigquery.storage.v1.BigQueryRead service, reached at bigquerystorage.googleapis.com:443 over TLS, with the main methods CreateReadSession and ReadRows (server streaming).

Requests are built with TsgcGRPCBigQueryCreateReadSessionRequest, which sets Parent (projects/<id>), ReadSession.Table (projects/<id>/datasets/<ds>/tables/<tbl>), ReadSession.DataFormat (1 = AVRO, 2 = ARROW) and MaxStreamCount. The session response carries the stream names, which are then read with the server-streaming ReadRows method.

The example below authenticates with a service-account JWT, wires a TsgcGRPCClient over a TsgcHTTP2Client to the BigQuery Storage host, sets the authorization Bearer metadata and calls CreateReadSession on a table using the AVRO data format:

    oHTTP2 := TsgcHTTP2Client.Create(nil);
    oHTTP2.Host := 'bigquerystorage.googleapis.com';
    oHTTP2.Port := 443;
    oHTTP2.TLS := True;

    oGRPC := TsgcGRPCClient.Create(nil);
    oGRPC.Client := oHTTP2;

    // service-account JWT authentication
    oGRPC.GoogleCloudOptions.JWT.KeyFile := 'service-account.json';
    oGRPC.GoogleCloudOptions.JWT.API_Endpoint := 'https://bigquerystorage.googleapis.com/';
    oGRPC.DefaultMetadata.AddValue('authorization', 'Bearer ' + oGRPC.GoogleCloudOptions.JWT.Token);

    // build the typed request and call the method
    oRequest := TsgcGRPCBigQueryCreateReadSessionRequest.Create;
    try
      oRequest.Parent := 'projects/my-project-id';
      oRequest.ReadSession.Table := 'projects/my-project-id/datasets/my_dataset/tables/my_table';
      oRequest.ReadSession.DataFormat := 1; // 1 = AVRO, 2 = ARROW
      oRequest.MaxStreamCount := 1;
      oResponse := oGRPC.Call('google.cloud.bigquery.storage.v1.BigQueryRead', 'CreateReadSession', oRequest.ToBytes);
      ShowMessage(oResponse.DataString);
    finally
      oRequest.Free;
    end;