feat: Added backend routes to export workflows + datasets to google drive

[Sentiaus]

What changes were proposed in this PR?

This PR adds server-side Google Drive export endpoints for datasets and workflows.
This PR relates to

New endpoints:

• POST /dataset/{did}/drive-export — Exports a dataset version (specified by dvid or latest=true) as a zip archive directly to a Google Drive resumable upload session URI.
• POST /dataset/drive-export/file — Exports a single file from a dataset version to Drive.
• POST /workflow/{wid}/export/drive — Exports a workflow's JSON content to Drive.

Security:

• All three endpoints validate that the session URI starts with https://www.googleapis.com/upload/drive/ before making any outbound connection, preventing SSRF.
• Dataset endpoints check read access and the dataset's isDownloadable flag before streaming data.
• Workflow endpoint checks read access via WorkflowAccessResource.

Size limits:

• Dataset version export: capped at 500 MB (DRIVE_EXPORT_MAX_DATASET_BYTES). The zip is built in memory before the PUT to Drive; this limit will be lifted once chunked/streaming resumable upload is implemented.
• Single file export: 5 TB (DRIVE_EXPORT_MAX_FILE_BYTES) — Google Drive's resumable upload ceiling.
• Workflow export: capped at 500 MB (DRIVE_EXPORT_MAX_BYTES) since the JSON content is read into a byte array before upload.

Config (user-system.conf):

• Adds clientSecret and apiKey fields (empty by default, overridable via USER_SYS_GOOGLE_CLIENT_SECRET / USER_SYS_GOOGLE_API_KEY env vars) so the frontend can fetch them from GoogleAuthResource.

GoogleAuthResource:

• Adds GET /auth/google/drive/apikey endpoint returning UserSystemConfig.googleApiKey so the frontend can initialize the Google Picker without embedding credentials in the build.

Any related issues, documentation, discussions?

Part of the Google Drive export feature. The frontend PR (stacked on this one) wires up the UI.
#4240
#5721 (old PR. Decided to close as this is much cleaner)

How was this PR tested?

• DatasetResourceSpec — new integration tests (using MockTexeraDB + MockLakeFS testcontainers) cover:
  • Successful version export to Drive (verifying the PUT to the session URI is made)
  • Successful single-file export to Drive
  • Rejection of non-googleapis session URIs (400 Bad Request)
  • 403 when the dataset is not downloadable
  • 400 when both dvid and latest are provided (or neither)
• WorkflowResource endpoint tested manually (recompile + restart required to pick up the new route).
• SSRF validation verified: passing a non-https://www.googleapis.com/upload/drive/ URI returns 400.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Sonnet 4.6

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @Ma77Ball, @VuMartin, @carloea2
  You can notify them by mentioning @Ma77Ball, @VuMartin, @carloea2 in a comment.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 60.52632% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.79%. Comparing base (0a5693c) to head (cff5644).
⚠️ Report is 239 commits behind head on main.

Files with missing lines	Patch %	Lines
...rce/dashboard/user/workflow/WorkflowResource.scala	65.00%	7 Missing ⚠️
...ache/texera/service/resource/DatasetResource.scala	57.14%	4 Missing and 2 partials ⚠️
.../texera/web/resource/auth/GoogleAuthResource.scala	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6447      +/-   ##
============================================
+ Coverage     55.27%   58.79%   +3.51%     
- Complexity     2991     3385     +394     
============================================
  Files          1117     1120       +3     
  Lines         43258    43595     +337     
  Branches       4668     4713      +45     
============================================
+ Hits          23912    25630    +1718     
+ Misses        17938    16542    -1396     
- Partials       1408     1423      +15     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)
agent-service	34.36% <ø> (ø)		Carriedforward from 0a5693c
amber	66.54% <62.50%> (+8.74%)	⬆️
computing-unit-managing-service	9.97% <ø> (+9.97%)	⬆️
config-service	52.30% <ø> (+0.74%)	⬆️
file-service	60.45% <57.14%> (+1.42%)	⬆️
frontend	48.87% <ø> (ø)		Carriedforward from 0a5693c
notebook-migration-service	78.94% <ø> (+0.37%)	⬆️
pyamber	90.20% <ø> (ø)		Carriedforward from 0a5693c
python	90.76% <ø> (ø)		Carriedforward from 0a5693c
workflow-compiling-service	55.14% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 8 worse · ⚪ 7 noise (<±5%) · 0 without baseline

Compared against main c7473a1 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	412	0.251	22,744/35,085/35,085 us	🔴 +14.2% / 🔴 +147.9%
🔴	bs=100 sw=10 sl=64	914	0.558	107,059/134,537/134,537 us	🔴 +20.4% / 🔴 +37.6%
⚪	bs=1000 sw=10 sl=64	1,102	0.673	902,472/1,012,393/1,012,393 us	⚪ within ±5% / 🔴 +7.5%

Baseline details

Latest main c7473a1 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	412 tuples/sec	455 tuples/sec	847.3 tuples/sec	-9.5%	-51.4%
bs=10 sw=10 sl=64	MB/s	0.251 MB/s	0.278 MB/s	0.517 MB/s	-9.7%	-51.5%
bs=10 sw=10 sl=64	p50	22,744 us	19,911 us	11,621 us	+14.2%	+95.7%
bs=10 sw=10 sl=64	p95	35,085 us	33,454 us	14,154 us	+4.9%	+147.9%
bs=10 sw=10 sl=64	p99	35,085 us	33,454 us	19,069 us	+4.9%	+84.0%
bs=100 sw=10 sl=64	throughput	914 tuples/sec	1,003 tuples/sec	1,104 tuples/sec	-8.9%	-17.2%
bs=100 sw=10 sl=64	MB/s	0.558 MB/s	0.612 MB/s	0.674 MB/s	-8.8%	-17.2%
bs=100 sw=10 sl=64	p50	107,059 us	98,330 us	91,279 us	+8.9%	+17.3%
bs=100 sw=10 sl=64	p95	134,537 us	111,747 us	97,771 us	+20.4%	+37.6%
bs=100 sw=10 sl=64	p99	134,537 us	111,747 us	107,798 us	+20.4%	+24.8%
bs=1000 sw=10 sl=64	throughput	1,102 tuples/sec	1,103 tuples/sec	1,127 tuples/sec	-0.1%	-2.2%
bs=1000 sw=10 sl=64	MB/s	0.673 MB/s	0.673 MB/s	0.688 MB/s	0.0%	-2.2%
bs=1000 sw=10 sl=64	p50	902,472 us	908,979 us	902,553 us	-0.7%	-0.0%
bs=1000 sw=10 sl=64	p95	1,012,393 us	1,026,081 us	941,491 us	-1.3%	+7.5%
bs=1000 sw=10 sl=64	p99	1,012,393 us	1,026,081 us	979,025 us	-1.3%	+3.4%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,486.00,200,128000,412,0.251,22743.66,35085.16,35085.16
1,100,10,64,20,2188.24,2000,1280000,914,0.558,107058.69,134537.09,134537.09
2,1000,10,64,20,18144.75,20000,12800000,1102,0.673,902472.17,1012392.81,1012392.81

[Yicong-Huang]

Hi @Sentiaus, could you please make the PR title more informative? Also I am lack of context, there is no issue or discussion linked in the PR description, could you please add that and hopefully that can make the description shorter?

[Sentiaus]

@Yicong-Huang sorry, I was making a last revision to the PR. Added the discussion and previous PR that I just closed. This is related to the google drive export discussion.