Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **`generative-deepseek`** — New `GenerativeConfigFactory.Deepseek(...)` collection config and `GenerativeProviderFactory.Deepseek(...)` runtime provider for the `generative-deepseek` module. Requires Weaviate ≥ 1.36.19.
- **Incremental Backups** — `BackupCreateRequest.IncrementalBaseBackupId` names an existing backup to build on, so files unchanged since that backup are not copied again; `Backup.IncrementalBaseBackupId` surfaces it on `List()` and `GetStatus()`. `Create` throws `WeaviateVersionMismatchException` below 1.37.0, the documented feature floor. The server has accepted the field on create since 1.34.18, but only returns it on read from 1.37.6.
- **`apiEndpoint` on `Multi2VecGoogle`** — Both `VectorizerFactory.Multi2VecGoogle(...)` overloads take an optional `apiEndpoint`, which selects the Gemini API (`generativelanguage.googleapis.com`) instead of Vertex AI. Both `Multi2VecGoogleGemini(...)` overloads also gained an optional `dimensions`.

### Fixed

- **Backup `Size` Dropped on List** — `BackupClient.List()` discarded the size and incremental base id it had already parsed from the response, so `Backup.Size` was null for every listed backup. Both fields are now mapped.
- **Aggregate Zeros for Absent Values** — Int, Number, Boolean, Text and Date results, `Count` included, reported `0`, `0.0` or `false` where the server had sent no value at all. Every scalar is now presence-checked and left null when unset, so an empty aggregation is distinguishable from a genuine zero.
- **`Multi2VecGoogleGemini` Emitted a Non-Existent Module** — The vectorizer declared the module name `multi2vec-google-gemini`, which no Weaviate server provides, so a collection configured with it could never be created. `VectorizerFactory.Multi2VecGoogleGemini(...)` now emits `multi2vec-google` with the Gemini API endpoint.

### Changed

- **`Multi2VecGoogleGemini` is now an `[Obsolete]` shim over `Multi2VecGoogle`** — This breaks at runtime, not only at compile time: the factory now returns a `Multi2VecGoogle`, so `is`/`as`/pattern matches on `Multi2VecGoogleGemini` silently stop matching, and a `switch` with arms for both types no longer compiles (`CS8120: The switch case is unreachable`). Bind the result as `Multi2VecGoogle` or `VectorizerConfig`.
- **`Multi2VecGoogle.ProjectId` and `.Location` are no longer `required`** — Both getters changed from `string!` to `string?` so the Gemini endpoint, which is scoped to neither a project nor a region, can omit them. Callers with nullable reference types enabled may see CS8600/CS8601.
- **`Aggregate.Boolean` counts are nullable** — `PercentageTrue`, `PercentageFalse`, `TotalTrue` and `TotalFalse` changed from `double`/`long` to `double?`/`long?`.
- **`BackupCreateRequest` gained a trailing optional parameter** — Its primary constructor and `Deconstruct` went from six parameters to seven, so six-element positional deconstruction (`var (id, backend, inc, exc, cpu, comp) = req;`) no longer compiles; deconstruct seven elements instead.
- **Binary compatibility** — The signature changes above are source-compatible for callers using named arguments, but not binary-compatible: assemblies compiled against 1.1.0 throw `MissingMethodException` until recompiled.

### Removed

- **`ReplicationAsyncConfig.MaxWorkers` and `ReplicationAsyncConfig.AliveNodesCheckingFrequency`** — Both fields have been no-ops on the server since Weaviate 1.37.3 and are now removed from the user-facing model, the OpenAPI spec, and the generated DTO. Existing code that sets these properties will not compile after upgrading; no behavioral change results from the removal.
- **`Multi2VecGoogleGemini.Model`** — Removed along with the rest of the type's own members; the base `Multi2VecGoogle` declares the same setting as `ModelId`. Migrate `.Model` → `.ModelId`.

---

Expand Down Expand Up @@ -67,7 +88,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

#### Vectorizers

- **Audio Field Support** ([#302](https://github.com/weaviate/weaviate-csharp-client/pull/302)): `Multi2VecGoogle` and `Multi2VecGoogleGemini` vectorizers now support audio field configurations with configurable per-field weights.
- **Audio Field Support** ([#302](https://github.com/weaviate/weaviate-csharp-client/pull/302)): `Multi2VecGoogle` and `Multi2VecGoogleGemini` vectorizers now support audio field configurations with configurable per-field weights. (`Multi2VecGoogleGemini` is deprecated in favour of `Multi2VecGoogle` — see Unreleased.)

#### API Ergonomics

Expand Down Expand Up @@ -105,7 +126,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Weaviate 1.36 support: HFresh vector index, async replication config, property index deletion
- Critical fix: gRPC vector serialization no longer doubles dimensions for non-`float[]` vectors
- Opt-in structured logging via `ILoggerFactory`
- New vectorizers: `Multi2VecGoogleGemini` and `Multi2MultivecWeaviate`
- New vectorizers: `Multi2VecGoogleGemini` (never functional — see Unreleased) and `Multi2MultivecWeaviate`

### Added

Expand All @@ -119,7 +140,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

#### Vectorizers

- **Multi2VecGoogleGemini** ([#297](https://github.com/weaviate/weaviate-csharp-client/pull/297)): New vectorizer calling the Google Gemini API directly. Supports image, text, and video field weighting. No project ID or location required (unlike the Vertex AI variant). Defaults to `generativelanguage.googleapis.com`.
- **Multi2VecGoogleGemini** ([#297](https://github.com/weaviate/weaviate-csharp-client/pull/297)): New vectorizer calling the Google Gemini API directly. Supports image, text, and video field weighting. No project ID or location required (unlike the Vertex AI variant). Defaults to `generativelanguage.googleapis.com`. **Correction:** this vectorizer declared the module name `multi2vec-google-gemini`, which no Weaviate server provides, so it could never create a collection; fixed in Unreleased.
- **Multi2MultivecWeaviate** ([#291](https://github.com/weaviate/weaviate-csharp-client/pull/291)): Support for the `multi2multivec-weaviate` vectorizer, which produces multi-vector embeddings using Weaviate's built-in model.
- **Cohere Reranker `BaseURL`** ([#287](https://github.com/weaviate/weaviate-csharp-client/pull/287)): Added `BaseURL` property to `RerankerCohereConfig` and a corresponding parameter to `RerankerConfigFactory.Cohere()`, enabling self-hosted or regional Cohere endpoints.

Expand Down
2 changes: 1 addition & 1 deletion ci/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ services:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
ENABLE_MODULES: text2vec-transformers,text2vec-cohere,backup-filesystem,generative-dummy,generative-anyscale,reranker-dummy,reranker-cohere,text2vec-ollama,generative-ollama
ENABLE_MODULES: text2vec-transformers,text2vec-cohere,backup-filesystem,generative-dummy,generative-anyscale,reranker-dummy,reranker-cohere,text2vec-ollama,generative-ollama,multi2vec-google,generative-deepseek
Comment thread
g-despot marked this conversation as resolved.
BACKUP_FILESYSTEM_PATH: "/tmp/backups"
EXPORT_ENABLED: 'true'
EXPORT_DEFAULT_PATH: "/tmp/exports"
Expand Down
41 changes: 29 additions & 12 deletions docs/AGGREGATE_RESULT_ACCESSORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,11 +107,11 @@ if (quantityAgg != null)
Console.WriteLine($"Total quantity: {quantityAgg.Sum}");
}

// Access boolean aggregation
// Access boolean aggregation (its members are null when the server did not send them)
var inStockAgg = result.Boolean("inStock");
if (inStockAgg != null)
if (inStockAgg?.PercentageTrue is { } inStockPercentage)
{
Console.WriteLine($"In stock: {inStockAgg.PercentageTrue:P0}");
Console.WriteLine($"In stock: {inStockPercentage:P0}");
}

// Access date aggregation
Expand Down Expand Up @@ -157,9 +157,13 @@ if (result.TryGetText("category", out var category))
}

// Generic TryGet
if (result.TryGet<Aggregate.Boolean>("inStock", out var stockAgg))
if (
result.TryGet<Aggregate.Boolean>("inStock", out var stockAgg)
&& stockAgg.TotalTrue is { } inStock
&& stockAgg.TotalFalse is { } outOfStock
)
{
Console.WriteLine($"In stock: {stockAgg.TotalTrue}, Out of stock: {stockAgg.TotalFalse}");
Console.WriteLine($"In stock: {inStock}, Out of stock: {outOfStock}");
}
```

Expand Down Expand Up @@ -196,7 +200,11 @@ result.Match("field",
text: t => Console.WriteLine($"Text: {t.Count} occurrences"),
integer: i => Console.WriteLine($"Integer: sum={i.Sum}"),
number: n => Console.WriteLine($"Number: mean={n.Mean}"),
boolean: b => Console.WriteLine($"Boolean: {b.PercentageTrue:P0} true"),
boolean: b =>
{
if (b.PercentageTrue is { } percentage)
Console.WriteLine($"Boolean: {percentage:P0} true");
},
date: d => Console.WriteLine($"Date range: {d.Minimum} to {d.Maximum}")
);

Expand All @@ -205,7 +213,10 @@ var description = result.Match<string>("field",
text: t => $"Text with {t.TopOccurrences.Count} unique values",
integer: i => $"Integer ranging from {i.Minimum} to {i.Maximum}",
number: n => $"Number with mean {n.Mean:F2}",
boolean: b => $"Boolean: {b.PercentageTrue:P0} true",
boolean: b =>
b.PercentageTrue is { } percentage
? $"Boolean: {percentage:P0} true"
: "Boolean: percentage not returned",
date: d => $"Dates from {d.Minimum:d} to {d.Maximum:d}"
);

Expand All @@ -223,7 +234,10 @@ foreach (var (name, _) in result.Properties)
text: t => $"{name}: {t.Count} items, top: {t.TopOccurrences.FirstOrDefault()?.Value}",
integer: i => $"{name}: range [{i.Minimum}, {i.Maximum}], sum: {i.Sum}",
number: n => $"{name}: range [{n.Minimum:F2}, {n.Maximum:F2}], mean: {n.Mean:F2}",
boolean: b => $"{name}: {b.TotalTrue} true, {b.TotalFalse} false",
boolean: b =>
b.TotalTrue is { } trueCount && b.TotalFalse is { } falseCount
? $"{name}: {trueCount} true, {falseCount} false"
: $"{name}: boolean counts not returned",
date: d => $"{name}: {d.Minimum:d} to {d.Maximum:d}"
);

Expand Down Expand Up @@ -312,10 +326,13 @@ For boolean properties:
| Property | Type | Description |
|----------|------|-------------|
| `Count` | `long?` | Number of values |
| `TotalTrue` | `long` | Count of true values |
| `TotalFalse` | `long` | Count of false values |
| `PercentageTrue` | `double` | Percentage of true values (0-1) |
| `PercentageFalse` | `double` | Percentage of false values (0-1) |
| `TotalTrue` | `long?` | Count of true values |
| `TotalFalse` | `long?` | Count of false values |
| `PercentageTrue` | `double?` | Percentage of true values (0-1) |
| `PercentageFalse` | `double?` | Percentage of false values (0-1) |

Every member is nullable: null means the server did not send that value, which is not the same as
zero. Check before formatting — interpolating a null renders as an empty string.

### Aggregate.Date

Expand Down
78 changes: 78 additions & 0 deletions src/Weaviate.Client.Tests/Integration/TestCollectionAggregate.cs
Original file line number Diff line number Diff line change
Expand Up @@ -903,4 +903,82 @@ await collectionClient.Data.Insert(
break;
}
}

/// <summary>
/// Two ways a scalar goes missing, on one collection: a filter that matches nothing, and a
/// sub-metric the caller never asked for. Both leave the optional proto field unset, and the
/// client must report null rather than the zero of the field's type — a caller cannot tell a
/// returned 0 apart from a genuine aggregate of zeros.
/// </summary>
[Fact]
public async Task Test_OverAll_Absent_Metrics_Are_Null_Not_Zero()
{
var collectionClient = await CollectionFactory(
properties: new[]
{
Property.Text("text"),
Property.Int("int"),
Property.Number("float"),
}
);

await collectionClient.Data.Insert(
new
{
text = "one",
@int = 7,
@float = 7.5,
},
cancellationToken: TestContext.Current.CancellationToken
);

// Nothing matches: the metrics were requested, but there is nothing to compute them over.
var noMatches = await collectionClient.Aggregate.OverAll(
filters: Filter.Property("text").IsEqual("no-such-value"),
returnMetrics:
[
Metrics.ForProperty("int").Integer(maximum: true, mean: true, sum: true),
Metrics.ForProperty("float").Number(maximum: true, mean: true, sum: true),
],
cancellationToken: TestContext.Current.CancellationToken
);

Assert.Equal(0, noMatches.TotalCount);

var emptyInteger = Assert.IsType<Aggregate.Integer>(noMatches.Properties["int"]);
Assert.Null(emptyInteger.Maximum);
Assert.Null(emptyInteger.Mean);
Assert.Null(emptyInteger.Sum);

var emptyNumber = Assert.IsType<Aggregate.Number>(noMatches.Properties["float"]);
Assert.Null(emptyNumber.Maximum);
Assert.Null(emptyNumber.Mean);
Assert.Null(emptyNumber.Sum);

// Objects do match here, so the nulls come from the metric selection alone.
var unrequested = await collectionClient.Aggregate.OverAll(
returnMetrics:
[
Metrics.ForProperty("int").Integer(maximum: true),
Metrics.ForProperty("float").Number(mean: true),
],
cancellationToken: TestContext.Current.CancellationToken
);

Assert.Equal(1, unrequested.TotalCount);

var integer = Assert.IsType<Aggregate.Integer>(unrequested.Properties["int"]);
Assert.Equal(7, integer.Maximum);
Assert.Null(integer.Mean);
Assert.Null(integer.Median);
Assert.Null(integer.Minimum);
Assert.Null(integer.Mode);
Assert.Null(integer.Sum);

var number = Assert.IsType<Aggregate.Number>(unrequested.Properties["float"]);
Assert.Equal(7.5, number.Mean);
Assert.Null(number.Maximum);
Assert.Null(number.Minimum);
Assert.Null(number.Sum);
}
}
49 changes: 49 additions & 0 deletions src/Weaviate.Client.Tests/Integration/TestCollections.cs
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,55 @@ public async Task Collection_Creates_And_Retrieves_Generative_Config()
Assert.IsType<GenerativeConfig.Custom>(collection.GenerativeConfig);
}

/// <summary>
/// Tests that a generative-deepseek collection round-trips through a real server. The module
/// validates its own settings on create (temperature, both penalties and topP all have
/// ranges, and baseURL is parsed), so a misspelt key or a wrong type shows up here rather
/// than silently falling back to a default.
/// </summary>
/// <remarks>
/// Integral floats only; the server drops fractional floats for this module on named-vector
/// classes. See PR #368. Fractional values are covered by the unit test instead.
/// </remarks>
[Fact]
public async Task Collection_Creates_And_Retrieves_GenerativeDeepseek_Config()
{
RequireModule("generative-deepseek");

// Arrange
var collectionClient = await CollectionFactory(
properties: [Property.Text("Name")],
generativeConfig: Configure.Generative.Deepseek(
model: "deepseek-chat",
temperature: 1,
maxTokens: 2048,
frequencyPenalty: 0,
presencePenalty: 0,
topP: 1,
baseURL: "https://api.deepseek.com",
stop: ["\n\n"]
)
);

// Act
var collection = await _weaviate
.Collections.Use(collectionClient.Name)
.Config.Get(TestContext.Current.CancellationToken);

// Assert
Assert.NotNull(collection);
var deepseek = Assert.IsType<GenerativeConfig.Deepseek>(collection.GenerativeConfig);
Assert.Equal("deepseek-chat", deepseek.Model);
Assert.Equal(1, deepseek.Temperature);
Assert.Equal(2048, deepseek.MaxTokens);
Assert.Equal(0, deepseek.FrequencyPenalty);
Assert.Equal(0, deepseek.PresencePenalty);
Assert.Equal(1, deepseek.TopP);
Assert.Equal("https://api.deepseek.com", deepseek.BaseURL);
Assert.NotNull(deepseek.Stop);
Assert.Equal(["\n\n"], deepseek.Stop);
}

/// <summary>
/// Tests that test collections export
/// </summary>
Expand Down
95 changes: 95 additions & 0 deletions src/Weaviate.Client.Tests/Integration/TestVectorizers.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
namespace Weaviate.Client.Tests.Integration;

using Weaviate.Client.Models;

/// <summary>
/// The vectorizer module tests class. Covers vectorizer configurations end to end, i.e. that the
/// module name and settings the client emits are ones a real server accepts.
/// </summary>
/// <seealso cref="IntegrationTests"/>
public class TestVectorizers : IntegrationTests
{
/// <summary>
/// Tests that the Google Gemini multimodal factory produces a collection the server accepts.
/// </summary>
[Fact]
public async Task Test_Multi2VecGoogleGemini_Creates_Collection()
{
RequireModule("multi2vec-google");
// The module is on every lane, but its Gemini path (apiEndpoint) only lands in 1.34.20;
// older builds still demand the Vertex projectId/location this factory omits.
RequireVersion("1.34.20");

var collection = await CollectionFactory(
name: "TestMulti2VecGoogleGemini",
properties: [Property.Text("text"), Property.Blob("image")],
vectorConfig: Configure.Vector(
"default",
v =>
v.Multi2VecGoogleGemini(
imageFields: ["image"],
textFields: ["text"],
dimensions: 512
)
)
);

var config = await collection.Config.Get(
cancellationToken: TestContext.Current.CancellationToken
);

Assert.NotNull(config);
var vectorizer = config.VectorConfig["default"].Vectorizer;
var google = Assert.IsType<Vectorizer.Multi2VecGoogle>(vectorizer);

Assert.Equal("multi2vec-palm", google.Identifier);
Assert.Equal("generativelanguage.googleapis.com", google.ApiEndpoint);
Assert.Equal(512, google.Dimensions);
// The factory has no vectorizeClassName: nothing is sent, nothing stored or defaulted back.
Assert.Null(google.VectorizeCollectionName);
Assert.NotNull(google.ImageFields);
Assert.Equal(["image"], google.ImageFields);
Assert.NotNull(google.TextFields);
Assert.Equal(["text"], google.TextFields);
// Vertex-only settings: the Gemini API has neither, and the server must not echo them.
Assert.Null(google.ProjectId);
Assert.Null(google.Location);
}

/// <summary>
/// Tests that the Vertex AI factory creates a collection with project id and location intact.
/// </summary>
[Fact]
public async Task Test_Multi2VecGoogle_Vertex_Creates_Collection()
{
RequireModule("multi2vec-google");

var collection = await CollectionFactory(
name: "TestMulti2VecGoogleVertex",
properties: [Property.Text("text"), Property.Blob("image")],
vectorConfig: Configure.Vector(
"default",
v =>
v.Multi2VecGoogle(
projectId: "my-project",
location: "us-central1",
imageFields: ["image"],
textFields: ["text"]
)
)
);

var config = await collection.Config.Get(
cancellationToken: TestContext.Current.CancellationToken
);

Assert.NotNull(config);
var vectorizer = config.VectorConfig["default"].Vectorizer;
var google = Assert.IsType<Vectorizer.Multi2VecGoogle>(vectorizer);

Assert.Equal("multi2vec-palm", google.Identifier);
Assert.Equal("my-project", google.ProjectId);
Assert.Equal("us-central1", google.Location);
Assert.Null(google.ApiEndpoint);
}
}
Loading
Loading