Skip to content

Pre-generate blurhash during image upload #19338

Description

@Antreesy

Might be a general enhancement for Talk: as files are shared and visible instantly, and usually one per user event, we can force metadata generation to not defer it to background job
Why it's needed in scope of classified conversations: we need to know a blurhash instantly to render a classified preview

Here's the patch that worked for me. Can't tell on the preformance implications regarding production:

--- a/lib/Share/Helper/FilesMetadataCache.php	(revision b9455b8c02830bdc562f1f4b4b6dfdfac94851ac)
+++ b/lib/Share/Helper/FilesMetadataCache.php	(date 1784906344689)
@@ -9,6 +9,7 @@
 
 namespace OCA\Talk\Share\Helper;
 
+use OCP\Files\Node;
 use OCP\FilesMetadata\Exceptions\FilesMetadataNotFoundException;
 use OCP\FilesMetadata\Exceptions\FilesMetadataTypeException;
 use OCP\FilesMetadata\IFilesMetadataManager;
@@ -40,12 +41,24 @@
 
 	/**
 	 * @param int $fileId
+	 * @param ?Node $node
 	 * @return array
 	 * @psalm-return array{width: int, height: int, blurhash?: string}
 	 * @throws FilesMetadataNotFoundException
 	 */
-	public function getImageMetadataForFileId(int $fileId): array {
-		if (!array_key_exists($fileId, $this->filesSizeData)) {
+	public function getImageMetadataForFileId(int $fileId, ?Node $node = null): array {
+		if ($node !== null && !isset($this->filesSizeData[$fileId]['blurhash'])) {
+			$this->cachePhotosSize($fileId, $this->filesMetadataManager->refreshMetadata(
+				$node,
+				IFilesMetadataManager::PROCESS_LIVE | IFilesMetadataManager::PROCESS_BACKGROUND,
+			));
+		} elseif (!array_key_exists($fileId, $this->filesSizeData)) {
 			try {
 				$this->cachePhotosSize($fileId, $this->filesMetadataManager->getMetadata($fileId, true));
 			} catch (FilesMetadataNotFoundException) {
--- a/lib/Chat/Parser/SystemMessage.php	(revision b9455b8c02830bdc562f1f4b4b6dfdfac94851ac)
+++ b/lib/Chat/Parser/SystemMessage.php	(date 1784906383227)
@@ -917,7 +917,8 @@
 
 		if ($isPreviewAvailable && str_starts_with($node->getMimeType(), 'image/')) {
 			try {
-				$sizeMetadata = $this->metadataCache->getImageMetadataForFileId($fileId);
+				$sizeMetadata = $this->metadataCache->getImageMetadataForFileId($fileId, $node);
 				if (isset($sizeMetadata['width'], $sizeMetadata['height'])) {
 					$data['width'] = (string)$sizeMetadata['width'];
 					$data['height'] = (string)$sizeMetadata['height'];
@@ -1085,7 +1086,8 @@
 		// If a preview is available, check if we can get the dimensions of the file from the metadata API
 		if ($isPreviewAvailable && str_starts_with($node->getMimeType(), 'image/')) {
 			try {
-				$sizeMetadata = $this->metadataCache->getImageMetadataForFileId($fileId);
+				$sizeMetadata = $this->metadataCache->getImageMetadataForFileId($fileId, $node);
 				if (isset($sizeMetadata['width'], $sizeMetadata['height'])) {
 					$data['width'] = (string)$sizeMetadata['width'];
 					$data['height'] = (string)$sizeMetadata['height'];

Originally posted by @Antreesy in #18679

Sounds good, I had 2 other ideas related:

  1. Blurhash could be calculated during the upload already (a listener on the file created event when coming from talk or when path is Talk/ or Talk/…/Draft)

  2. Change the upload/share timeline:

    1. When you drop a file we start uploading to Draft/ (since its not shared yet)
    2. We can already start blurhash and everything
    3. On Post we move to shared folder and upload+blurhash is ready already

Originally posted by @nickvergessen in #18679

Discussed internally (for Web):

  • blurhash library already present to encode string to blurred preview
  • Could be done before uploading file on client side, to not wait for server to re-validate it
  • blurhash, width, height all available and can be passed from client
  • only those three need on client side for instant showing blurred preview
  • need a way to provide this data to server (together with talkMetadata? as files_sharing API payload)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions